tag:blogger.com,1999:blog-2202766463251903159.post5979699925759633905..comments2024-03-28T14:53:31.889-07:00Comments on Sunil's Notes: Saving complex object in elasticsearch as output of MapReduce programSunil Patilhttp://www.blogger.com/profile/16075929903536310688noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-2202766463251903159.post-40241598615472687322016-05-16T04:01:17.000-07:002016-05-16T04:01:17.000-07:00Hi Sunil
I ran the code of yours downloaded from G...Hi Sunil<br />I ran the code of yours downloaded from GitHub....but I am getting this error<br />"java.lang.StringIndexOutOfBoundsException: String index out of range: -1".....I cant understand why it is coming.....can you please help me with this?<br />Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-2202766463251903159.post-85858378032143594412016-01-16T11:59:15.903-08:002016-01-16T11:59:15.903-08:00@Unknown If your document has a particular field t...@Unknown If your document has a particular field that you would like to use as id then you can set that using <br /><br />conf.set("es.mapping.id", "fieldname");<br /><br />Ex. In this case i can configure firstName to be the _id for record and make sure that no 2 players have same first name by setting<br /><br />conf.set("es.mapping.id", "firstName");<br /><br />Sunil Patilhttps://www.blogger.com/profile/16075929903536310688noreply@blogger.comtag:blogger.com,1999:blog-2202766463251903159.post-33769335968167562992016-01-16T11:56:17.673-08:002016-01-16T11:56:17.673-08:00@featherSoft Api: You might want to use a reducer ...@featherSoft Api: You might want to use a reducer and move code for generating output to reducer. For example mapper read 3 lines <br />A,B,C <br />B,D,E<br />A,B,C<br /><br />You want unique records based on first column A, so your mapper can publish A as key and (A,B,C) as value your reducer will get (A, (A,B,C), (A,B,C)). Now you can combine all the values for A and then at the end insert one record in ES. <br /><br />The reason for reducing is it will make sure that all the records with Key equal to A finally get consolidated to single machineSunil Patilhttps://www.blogger.com/profile/16075929903536310688noreply@blogger.comtag:blogger.com,1999:blog-2202766463251903159.post-4695424562587600532016-01-11T05:41:47.182-08:002016-01-11T05:41:47.182-08:00But this map reduce program generates duplicate e...But this map reduce program generates duplicate entries. My file has 180 fields and 1 crore records. Can I trust map reduce program to insert data into ElasticSearch ?Anonymoushttps://www.blogger.com/profile/16414770473736207051noreply@blogger.comtag:blogger.com,1999:blog-2202766463251903159.post-56791136189317934462016-01-09T01:07:19.483-08:002016-01-09T01:07:19.483-08:00How can give _id value from the file as a serial n...How can give _id value from the file as a serial no in this Program ?<br />Anonymoushttps://www.blogger.com/profile/16414770473736207051noreply@blogger.com