-
Connect to your Amazon EC2 instance using SSH and execute following command to download ElasticSearch v 90.9 on my instance
It took less than a minute to download the elasticsearch binary on the instancewget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.9.tar.gz -
Next i executed following command to unzip the elasticsearch-0.90.9.tar.gz
tar -xf elasticsearch-0.90.9.tar.gz -
After extracting the elasticsearch binaries start it by executing following command
elasticsearch-0.90.9/bin/elasticsearch –f -
After starting the ElasticSearch instance i could use the ES REST API from the SSH console on the same instance
curl -XGET http://localhost:9200/_search -
But since i want to access it from my local machine/outside the instance i had to open the 9200 port on that instances firewall that i could do by changing the security group and adding 9200 and 9300 ports to it
- Then you can use the public DNS for your EC2 instance and query the REST instance from your local machine like this
curl -XGET http://<yourinstancename>.us-west-1.compute.amazonaws.com:9200/_search
Installing ElasticSearch on Amazon EC2
I wanted to set up ElasticSearch on Amazon EC2 and these are the steps that i followed to do that
Using Elastic Search to implement Locate US/ Geo Saptial Search
Elastic Search has nice support for geo spatial search. I wanted to try that out. So i started by creating a index called testgeo and i did add addresses of few walmart locations in it. Then i used Elastic Search to figure out what store location is close to my location. I followed these steps to implement this sample
- First i did create a new Index named testgeo like this, while creating index i marked location field as geo_type, this makes it possible to make geo distance queries.
curl -XPUT 'localhost:9200/testgeo' -d '{ "mappings": { "walmart": { "properties": { "shopName": { "type": "string" }, "address": { "properties": { "city": { "type": "string" }, "state": { "type": "string" }, "streetName": { "type": "string" }, "location": { "type": "geo_point" } } } } } } }' -
Then i did add few locations to my index for each of the location i used the latitude, longitude for address of the location and stored it in my testgeo index
curl -XPOST 'localhost:9200/testgeo/walmart' -d '{ "storeName": "Walmart Supercenter - Marina", "address": { "streetName": "150 Beach Rd", "cityName": "Marina", "state": "CA", "zipCode": 93933, "location": [ -121.800565, 36.69359 ] } }' curl -XPOST 'localhost:9200/testgeo/walmart' -d '{ "storeName": "Walmart Mkt - Modesto", "address": { "streetName": "\"1421 Coffee Rd", "cityName": "Modesto", "state": "CA", "zipCode": 95355, "location": [ -120.976622, 37.664054 ] } }' curl -XPOST 'localhost:9200/testgeo/walmart' -d '{ "storeName": "Walmart Supercenter - Patterson", "address": { "streetName": "\"1030 Sperry Ave", "cityName": "Patterson", "state": "CA", "zipCode": 95363, "location": [ -121.142528, 37.464052 ] } }' -
Once my index data is in place i could query the index with type equal to geo_distance, and i had to give a location from which i want to search all the stores in 100 KM of the location
curl -XPOST 'localhost:9200/testgeo/repair/_search?pretty=true' -d ' { "query": { "filtered": { "query": { "match_all": {} }, "filter": { "geo_distance": { "distance": 100, "distance_unit": "km", "address.location": { "lat": 37.53, "lon": -121.97 } } } } } }'
Using SalesForce.com REST API
I was trying to use salesforce.com REST API for getting data. So i started following the http://wiki.developerforce.com/page/Integrating_Java_Spring_Apps_on_Heroku_with_Force.com_REST_APIs document. But after i followed it i kept getting 
Once i set up the oAUth settings properly the sample started working
Unable to get access token exception, more details here
at com.force.sdk.oauth.connector.ForceOAuthConnector.createTokenInternal(ForceOAuthConnector.java:196)
2013-11-28T05:58:29.294965+00:00 app[web.1]: at org.springframework.security.authentication.AbstractAuthenticationManager.authenticate(AbstractAuthenticationManager.java:48)
2013-11-28T05:58:29.294965+00:00 app[web.1]: 05:58:29,267 ERROR OAuthAuthenticationProvider:82 - Unable to get access token
2013-11-28T05:58:29.294965+00:00 app[web.1]: java.io.IOException: Missing refresh token on response
2013-11-28T05:58:29.294965+00:00 app[web.1]: at org.springframework.security.authentication.ProviderManager.doAuthentication(ProviderManager.java:130)
2013-11-28T05:58:29.294965+00:00 app[web.1]: at com.force.sdk.springsecurity.OAuthAuthenticationProvider.authenticate(OAuthAuthenticationProvider.java:78)
2013-11-28T05:58:29.300136+00:00 heroku[router]: at=info method=GET path=/_auth?code=aPrxkpHJh6tvoT6FsBbmu3.hEs0MRNTo0tj3P9AWu2oQ3aJeF7rXY._JA1wepFrePIgKkBuF4w%3D%3D&state=https%3A%2F%2Fdry-refuge-2742.herokuapp.com%2Fpeople%2F host=dry-refuge-2742.herokuapp.com fwd="206.218.52.33" dyno=web.1 connect=1ms service=446ms status=401 bytes=1083
2013-11-28T05:58:29.295172+00:00 app[web.1]: at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:355)
2013-11-28T05:58:29.294965+00:00 app[web.1]: at com.force.sdk.oauth.connector.ForceOAuthConnector.getAccessToken(ForceOAuthConnector.java:145)
2013-11-28T05:58:29.294965+00:00 app[web.1]: at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:199)
2013-11-28T05:58:29.295172+00:00 app[web.1]: at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:355)
2013-11-28T05:58:29.294965+00:00 app[web.1]: at com.force.sdk.springsecurity.AuthenticationProcessingFilter.doFilter(AuthenticationProcessingFilter.java:109)
2013-11-28T05:58:29.294965+00:00 app[web.1]: at com.force.sdk.springsecurity.AuthenticationProcessingFilter.attemptAuthentication(AuthenticationProcessingFilter.java:138)
2013-11-28T05:58:29.295172+00:00 app[web.1]: at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilt
er(SecurityContextPersistenceFilter.java:79)
2013-11-28T05:58:29.295172+00:00 app[web.1]: at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
2013-11-28T05:58:29.295172+00:00 app[web.1]: at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
2013-11-28T05:58:29.295172+00:00 app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
2013-11-28T05:58:29.295172+00:00 app[web.1]: at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
2013-11-28T05:58:29.295172+00:00 app[web.1]: at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:355)
2013-11-28T05:58:29.295416+00:00 app[web.1]: at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
2013-11-28T05:58:29.295172+00:00 app[web.1]: at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:149)
2013-11-28T05:58:29.295416+00:00 app[web.1]: at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
2013-11-28T05:58:29.295416+00:00 app[web.1]: at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
2013-11-28T05:58:29.295416+00:00 app[web.1]: at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
2013-11-28T05:58:29.295172+00:00 app[web.1]: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
2013-11-28T05:58:29.295416+00:00 app[web.1]: at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
2013-11-28T05:58:29.295416+00:00 app[web.1]: at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
2013-11-28T05:58:29.295946+00:00 app[web.1]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
2013-11-28T05:58:29.295946+00:00 app[web.1]: at java.lang.Thread.run(Thread.java:679)
2013-11-28T05:58:29.295416+00:00 app[web.1]: at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
2013-11-28T05:58:29.295416+00:00 app[web.1]: at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
2013-11-28T05:58:29.295946+00:00 app[web.1]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
2013-11-28T05:58:29.295416+00:00 app[web.1]: at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
2013-11-28T05:58:29.295416+00:00 app[web.1]: at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
2013-11-28T05:58:24.518283+00:00 heroku[router]: at=info method=GET path=/people/ host=dry-refuge-2742.herokuapp.com fwd="206.218.52.33" dyno=web.1 connect=1ms service=5ms status=302 bytes=0
- Full Access(full)
- Perform request on your behalf at any time (refresh_token)
Using Salesforce SOAP API from Java Code
Recently i had to develop a portlet that talks to SalesForce.com and retrieves some data. I decided to use the SalesForce SOAP API for that, i followed these steps
- First step is to get your salesforce.com security token, follow steps as described in Generating Security Token for SalesForce.com post for that
- Second step is to login into your salesforce.com account and download the Enterprise .wsdl, Follow steps 1-3 from Accessing SalesForce data using SOAP Service - SoapUi post for that
- You should use the Force.com web services connector (WSC) which has helper code that simplify working with Force.com SOAP service.
- Once you have WSC project downloaded on your machine and complied into jar copy it in a directory and then also copy your enterprise.wsdl in the same directory, then execute
It will generate enterprise.jar file for youjava -classpath target/force-wsc-29.0.0-jar-with-dependencies.jar com.sforce.ws.tools.wsdlc enterprise.wsdl enterprise.jar - Now create a Java Project and add both enterprise.jar and force-wsc-29.0.0-jar-with-dependencies.jar in your classpath
- Last step is to create SFDClient.java class like this Replace userId, password and securitytoken in this file with the values you have
- When you run the SFDClient.java class it will display list of contacts in your salesforce.com account
Accessing SalesForce data using SOAP Service - SoapUi
Recently i was trying to figure out how to access SalesForce data from outside. I had two options either to use REST or use SOAP API for accessing the data. I tried both and these are my notes for accessing SalesForce Soap API using Soap client such as SoapUi.
- First thing that you would want to do is generate security token for salesforce. Follow these steps for generating security token
-
Once you have security token, login into the SalesForce console to download the WSDL, Click on Develop -< API and you will see page where you can download all the WSDL's, Download the Enterprise WSDL
-
Now create new SOAP Ui project using the Enterprise WSDL you just downloaded
You will notice that it generates bunch of SOAP operations
-
In order to use the SOAP API you will have to first send a login request to SalesForce and get sessionid once you have it you can use that for making subsequent calls. First call the Login service like this
You should replace replacerwitheuserid with your userId (email id that you use for login) and also replace replacewithpassword with your password and use the security token that you got in email to replace replacewithsecuritytoken. Make a request and you will get a response which looks like this<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:enterprise.soap.sforce.com"> <soapenv:Header> </soapenv:Header> <soapenv:Body> <urn:login> <urn:username>replacerwitheuserid</urn:username> <urn:password>replacewithpasswordreplacewithsecuritytoken</urn:password> </urn:login> </soapenv:Body> </soapenv:Envelope>
You will need the value of<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns="urn:enterprise.soap.sforce.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <soapenv:Body> <loginResponse> <result> <metadataServerUrl>https://na15.salesforce.com/services/Soap/m/29.0/00Di0000000iKns</metadataServerUrl> <passwordExpired>false</passwordExpired> <sandbox>false</sandbox> <serverUrl>https://na15.salesforce.com/services/Soap/c/29.0/00Di0000000iKns</serverUrl> <sessionId>sessionId</sessionId> <userId>userId</userId> <userInfo> <accessibilityMode>false</accessibilityMode> <currencySymbol>$</currencySymbol> <orgAttachmentFileSizeLimit>5242880</orgAttachmentFileSizeLimit> <orgDefaultCurrencyIsoCode>USD</orgDefaultCurrencyIsoCode> <orgDisallowHtmlAttachments>false</orgDisallowHtmlAttachments> <orgHasPersonAccounts>false</orgHasPersonAccounts> <organizationId>xxxx</organizationId> <organizationMultiCurrency>false</organizationMultiCurrency> <organizationName>Self</organizationName> <profileId>xxxx</profileId> <roleId xsi:nil="true"/> <sessionSecondsValid>7200</sessionSecondsValid> <userDefaultCurrencyIsoCode xsi:nil="true"/> <userEmail>xxxxx</userEmail> <userFullName>Sunil Patil</userFullName> <userId>xxxxx</userId> <userLanguage>en_US</userLanguage> <userLocale>en_US</userLocale> <userName>sxxxxxx</userName> <userTimeZone>America/Los_Angeles</userTimeZone> <userType>Standard</userType> <userUiSkin>Theme3</userUiSkin> </userInfo> </result> </loginResponse> </soapenv:Body> </soapenv:Envelope>sessionIdelement and value ofserverUrlin subsequent requests. so note it down -
Next assume that you want to execute query request that will give you list of all the contacts using
SELECT Id, FirstName, LastName From Contactquery, so use XML that looks like this for the soap request. Replace the replacewithyoursessionid with thesessionIdthat you got from the login request and also change the URL where SOAP request is made to value ofserverUrl
You will get response with list of contacts<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:enterprise.soap.sforce.com"> <soapenv:Header> <urn:SessionHeader> <urn:sessionId>replacewithyoursessionid</urn:sessionId> </urn:SessionHeader> </soapenv:Header> <soapenv:Body> <urn:query> <urn:queryString> SELECT Id, FirstName, LastName From Contact< /urn:queryString> </urn:query> </soapenv:Body> </soapenv:Envelo
Generating Security Token for SalesForce.com
If you want to use SalesForce SOAP API then you will need security token. If you dont have the security token SalesForce will generate Soap fault like this
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:sf="urn:fault.enterprise.soap.sforce.com"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<soapenv:Fault>
<faultcode>sf:LOGIN_MUST_USE_SECURITY_TOKEN</faultcode>
<faultstring>LOGIN_MUST_USE_SECURITY_TOKEN: Invalid username, password, security token; or user locked out. Are you at a new location? When accessing Salesforce--either via a desktop client or the API--from outside of your company’s trusted networks, you must add a security token to your password to log in. To receive a new security token, log in to salesforce.com at http://login.salesforce.com and click Setup | My Personal Information | Reset Security Token.</faultstring>
<detail>
<sf:LoginFault xsi:type="sf:LoginFault">
<sf:exceptionCode>LOGIN_MUST_USE_SECURITY_TOKEN</sf:exceptionCode>
<sf:exceptionMessage>Invalid username, password, security token; or user locked out. Are you at a new location? When accessing Salesforce--either via a desktop client or the API--from outside of your company’s trusted networks, you must add a security token to your password to log in. To receive a new security token, log in to salesforce.com at http://login.salesforce.com and click Setup | My Personal Information | Reset Security Token.</sf:exceptionMessage>
</sf:LoginFault>
</detail>
</soapenv:Fault>
</soapenv:Body>
</soapenv:Envelope>
- Login into your Salesforce account
- Click on your use name and you will get drop down like this, click on My Settings sub menu
- On the next page you will get submenu on the side click on Personal side menu, once its expanded click on Reset my security token link
-
Once your on the Reset My Security Token button and it will trigger security token generation process
- In few minutes it will send email to your address with security token, use that for the subsequent SOAP calls
Customizing the JSON output generated by Jackson
Recently i was using Jackson JSON Processor for converting java object into JSON. When i did that i noticed JackSon was storing date in the long format i.e. it was converting date.getTime() and using that long value in JSON instead of formatted date. This is how my Contact object looks like
Note: please ignore line 32
@JsonSerialize(using = CustomDateSerializer.class) for now. I had following code to create object of Contact and converting and then using Jackson to convert it to String and write it on console
This was the output that got generated
{"firstName":"Sachin","lastName":"Tendulkar","dateOfBirth":60065349443063}
dd-MMM-yyyy format. In order to do that i started by creating CustomDateSerializer class like this
Then i had to change the Contact.java class to add @JsonSerialize(using = CustomDateSerializer.class) annotation to getDateOfBirth() field, i could attach it to any other date field that i want. Now when i run JSONTester this the output that it generates
{"firstName":"Sachin","lastName":"Tendulkar","dateOfBirth":"24-Apr-1973"}
Using Mule to get Geo Code for address
For last few days i am playing around with Mule, its really cool integration framework. You can use it to build integration applications by using small amount of code. I had this requirement where given address i wanted to get the Geo Coding information for that address, I wanted to use Google Maps API for getting GeoCode information for the address.
The way Google Maps API works is it has a REST API, it takes address as query string and it returns JSON structure. Ex. If you copy paste this URL in the browser
http://maps.googleapis.com/maps/api/geocode/json?address=3055 Oak Road,Walnut Creek,CA,94597&sensor=false
You will get a JSON structure with longitude and latitude embedded in lot of other information
{
"results" : [
{
"address_components" : [
{
"long_name" : "3055",
"short_name" : "3055",
"types" : [ "street_number" ]
},
{
"long_name" : "Oak Road",
"short_name" : "Oak Rd",
"types" : [ "route" ]
},
{
"long_name" : "Walnut Creek",
"short_name" : "Walnut Creek",
"types" : [ "locality", "political" ]
},
{
"long_name" : "Contra Costa County",
"short_name" : "Contra Costa County",
"types" : [ "administrative_area_level_2", "political" ]
},
{
"long_name" : "California",
"short_name" : "CA",
"types" : [ "administrative_area_level_1", "political" ]
},
{
"long_name" : "United States",
"short_name" : "US",
"types" : [ "country", "political" ]
},
{
"long_name" : "94597",
"short_name" : "94597",
"types" : [ "postal_code" ]
}
],
"formatted_address" : "3055 Oak Road, Walnut Creek, CA 94597, USA",
"geometry" : {
"bounds" : {
"northeast" : {
"lat" : 37.9302338,
"lng" : -122.0582288
},
"southwest" : {
"lat" : 37.9302299,
"lng" : -122.0582466
}
},
"location" : {
"lat" : 37.9302299,
"lng" : -122.0582466
},
"location_type" : "RANGE_INTERPOLATED",
"viewport" : {
"northeast" : {
"lat" : 37.93158083029149,
"lng" : -122.0568887197085
},
"southwest" : {
"lat" : 37.9288828697085,
"lng" : -122.0595866802915
}
}
},
"types" : [ "street_address" ]
}
],
"status" : "OK"
}
[
-122.0582466,
37.9302299
]
[lng,lat] format, some open source JavaScript Mapping frameworks understand the GeoJSON format.
I built a simple Mule integration application which takes STREETLINE1, CITY, STATE, ZIPCODE as argument and returns geo location in GeoJSON format like this [lng,lat]. I followed these steps to build the application
-
First i built a Mule Flow which looks like this
In this flow i have a composite input source that could take input on either VM transport or HTTP transport and then uses the values submitted by user to make a request to Google Maps API. Once the response is returned it uses DataMapper to convert the big response that Google Maps API returns into simple GeoJSON response.
- This is how the Data Mapping file looks like
If you look at the script of this file it looks like this
How Scala code gets compiled into Java
For last few days i am learning Scala + play framework. Since i have been using Java for long time now, one that always helps is to convert Scala code into Java Code and see what is going on under the hood. I follow these steps
- First i create a new Scala File say Contact.scala which has one Scala class like this
class Contact{ var firstName ="" private var lastName = "" } -
Then go to the directory that has Contact.Scala and compile it using
scalac Contact.scala, The scala compiler will create .class files for every class in that .sclaa file in appropriate package. In my case i have only one class in my Contact.scala so it creates Contact.class -
Next use javap Contact and and i can see following output being generated on the command line
Compiled from "Person.scala" public class Contact extends java.lang.Object{ private java.lang.String firstName; private java.lang.String lastName; public java.lang.String firstName(); public void firstName_$eq(java.lang.String); private java.lang.String lastName(); private void lastName_$eq(java.lang.String); public Contact(); }
Using AngularJs in Worklight/PhoneGap application
Angular.js is a JavaScript MVC framework, that makes development of HTML applications easy. I wanted to figure out how to use it for developing Worklight application so i followed these steps to build simple Hello AngularJs application
- First create a WorkLight application using WorkLight wizard, make sure that it works
- Next make changes in the index.html or entry page of your application to include angular.js from Google CDN and also add
<p>Hello {{'World'.length}}</p> to test if AngularJs template is working
<!DOCTYPE HTML>
<html>
<head>
<meta charset="UTF-8">
<title>HelloWorld</title>
<meta name="viewport" content="width=device-width,
initial-scale=1.0, maximum-scale=1.0,
minimum-scale=1.0, user-scalable=0">
<link rel="shortcut icon" href="images/favicon.png">
<link rel="apple-touch-icon" href="images/apple-touch-icon.png">
<link rel="stylesheet" href="css/HelloWorld.css">
<script>window.$ = window.jQuery = WLJQ;</script>
</head>
<body id="content" style="display: none;" ng-app>
<h1>Hello Angularjs</h1>
<p>Hello {{'World'.length}}</p>
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.0.6/angular.min.js" />
<script src="js/initOptions.js"></script>
<script src="js/HelloWorld.js"></script>
<script src="js/messages.js"></script>
</body>
</html
Subscribe to:
Posts (Atom)