GC Overhead limit exceeded error in Hadoop Enamul Karim https://conferences.xsede.org/c/message_boards/find_message?p_l_id=&messageId=2804194 2021-09-07T21:27:08Z 2021-09-07T21:27:08Z

I am trying to run a MapReduce application in a single node machine and utilize its multicore to act as mapper and reducer. I am setting it in the following way:<br /><br />Configuration conf = new Configuration();<br />conf.set(&#034;mapreduce.tasktracker.map.tasks.maximum&#034;, String.valueOf(NO_OF_MAPPERS));<br />conf.set(&#034;mapreduce.tasktracker.reduce.tasks.maximum&#034;, String.valueOf(NO_OF_REDUCERS));<br /><br />I am not sure whether I am doing it right because I don&#039;t see any performance increase by using a higher number of mappers and reducers. Can anyone help me? Any help is highly appreciated.

Enamul Karim 2021-09-07T21:27:08Z Enamul Karim https://conferences.xsede.org/c/message_boards/find_message?p_l_id=&messageId=2523438 2020-07-30T09:13:38Z 2020-07-30T09:05:57Z

I am working on a Hadoop Map/Reduce application. In one of the experiments with a large data set, I got an error saying that GC overhead limit exceeded. Can anyone tell me what could be the reason and how can I solve it in Hadoop?

Enamul Karim 2020-07-30T09:05:57Z