Predictive Analytics. Container killed by YARN for exceeding memory limits

[former_member1238168]

Hi,

I am trying to train a model in Predictive Analytics using Native Spark Modeling (I have already trained another small model and it worked fine), but the training fails and I get the following error message in the IDBM logs:

ERROR YarnScheduler:74 - Lost executor 1 on 404-39-c02.rt1.altiscale.com: Container killed by YARN for exceeding memory limits. 1.6 GB of 1.5 GB physical memory used.

I understand this is happening because I don't have enough memory, right? do I need to modify a configuration file to fix this?

Thank you

[achab · Accepted solution]

Hello Rafael, can I suggest you to raise a support ticket on this? I do not have the expertise to help here. Thanks & kind regards Antoine