By: Tiago Fernandes user 28 Jun 2021 at 11:44 a.m. CDT

4 Responses

Good afternoon. After performing a script update to a select few custom UMA policies in our installation, several policies began appearing in red and throwing errors. The most peculiar is that policies that were unrelated to the changes also showed the same error. I found another ticket mentioning this sort of error at https://support.gluu.org/outages/8650/out-of-memory-metaspace/ But since we are doing a kubernetes installation using the docker images, I am not aware of how we can modify these memory values during (or even after) installation. Any help will be appreciated. ``` org.gluu.exception.PythonException: Failed to load python file 'java.io.ByteArrayInputStream@2afda5fe' at org.gluu.service.PythonService.loadPythonScript(PythonService.java:184) at org.gluu.service.PythonService$Proxy$_$$_WeldClientProxy.loadPythonScript(Unknown Source) at org.gluu.service.custom.script.CustomScriptManager.createExternalTypeFromStringWithPythonException(CustomScriptManager.java:390) at org.gluu.service.custom.script.CustomScriptManager$Proxy$_$$_WeldSubclass.createExternalTypeFromStringWithPythonException(Unknown Source) at org.gluu.service.custom.script.CustomScriptManager.createExternalType(CustomScriptManager.java:358) at org.gluu.service.custom.script.CustomScriptManager.reloadCustomScriptConfigurations(CustomScriptManager.java:268) at org.gluu.service.custom.script.CustomScriptManager.reloadImpl(CustomScriptManager.java:157) at org.gluu.service.custom.script.CustomScriptManager.reload(CustomScriptManager.java:144) at org.gluu.service.custom.script.CustomScriptManager.reloadTimerEvent(CustomScriptManager.java:120) at org.gluu.service.custom.script.CustomScriptManager$Proxy$_$$_WeldSubclass.reloadTimerEvent(Unknown Source) at sun.reflect.GeneratedMethodAccessor156.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:95) at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:85) at org.jboss.weld.injection.MethodInvocationStrategy$SimpleMethodInvocationStrategy.invoke(MethodInvocationStrategy.java:168) at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:330) at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:308) at org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:286) at javax.enterprise.inject.spi.ObserverMethod.notify(ObserverMethod.java:124) at org.jboss.weld.util.Observers.notify(Observers.java:166) at org.jboss.weld.event.ObserverNotifier.notifySyncObservers(ObserverNotifier.java:285) at org.jboss.weld.event.ObserverNotifier.notify(ObserverNotifier.java:273) at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:177) at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:159) at org.jboss.weld.manager.BeanManagerImpl.fireEvent(BeanManagerImpl.java:632) at org.jboss.weld.util.ForwardingBeanManager.fireEvent(ForwardingBeanManager.java:104) at org.gluu.service.timer.TimerJob.execute(TimerJob.java:39) at org.gluu.service.timer.JobExecutionDelegate.execute(JobExecutionDelegate.java:31) at org.quartz.core.JobRunShell.run(JobRunShell.java:202) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) Caused by: Traceback (most recent call last): File "scim_protected_policy.py", line 19, in <module> File "/opt/jython/Lib/httplib.py", line 72, in <module> import socket File "/opt/jython/Lib/warnings.py", line 233, in warn warn_explicit(message, category, filename, lineno, module, registry, File "/opt/jython/Lib/warnings.py", line 272, in warn_explicit linecache.getlines(filename, module_globals) File "/opt/jython/Lib/linecache.py", line 40, in getlines return updatecache(filename, module_globals) File "/opt/jython/Lib/linecache.py", line 80, in updatecache stat = os.stat(fullname) File "/opt/jython/Lib/linecache.py", line 80, in updatecache stat = os.stat(fullname) java.lang.OutOfMemoryError: Metaspace java.lang.OutOfMemoryError: java.lang.OutOfMemoryError: Metaspace at org.python.core.Py.JavaError(Py.java:552) at org.python.core.Py.setException(Py.java:1544) at linecache$py.updatecache$5(/opt/jython/Lib/linecache.py:135) at linecache$py.call_function(/opt/jython/Lib/linecache.py) at org.python.core.PyTableCode.call(PyTableCode.java:171) at org.python.core.PyBaseCode.call(PyBaseCode.java:154) at org.python.core.PyFunction.__call__(PyFunction.java:423) at linecache$py.getlines$3(/opt/jython/Lib/linecache.py:40) at linecache$py.call_function(/opt/jython/Lib/linecache.py) at org.python.core.PyTableCode.call(PyTableCode.java:171) at org.python.core.PyBaseCode.call(PyBaseCode.java:154) at org.python.core.PyFunction.__call__(PyFunction.java:423) at warnings$py.warn_explicit$18(/opt/jython/Lib/warnings.py:301) at warnings$py.call_function(/opt/jython/Lib/warnings.py) at org.python.core.PyTableCode.call(PyTableCode.java:171) at org.python.core.PyBaseCode.call(PyBaseCode.java:308) at org.python.core.PyFunction.function___call__(PyFunction.java:471) at org.python.core.PyFunction.__call__(PyFunction.java:466) at org.python.core.PyFunction.__call__(PyFunction.java:456) at warnings$py.warn$17(/opt/jython/Lib/warnings.py:233) at warnings$py.call_function(/opt/jython/Lib/warnings.py) at org.python.core.PyTableCode.call(PyTableCode.java:171) at org.python.core.PyBaseCode.call(PyBaseCode.java:171) at org.python.core.PyFunction.__call__(PyFunction.java:434) at org.python.core.PyFunction.__call__(PyFunction.java:428) at org.python.core.Py.warning(Py.java:512) at org.python.core.Py.warning(Py.java:499) at org.python.core.PyTableCode.call(PyTableCode.java:175) at org.python.core.PyCode.call(PyCode.java:18) at org.python.core.imp.createFromCode(imp.java:436) at org.python.core.imp.createFromPyClass(imp.java:237) at org.python.core.imp.createFromPyClass(imp.java:205) at org.python.core.imp.loadFromSource(imp.java:657) at org.python.core.imp.find_module(imp.java:551) at org.python.core.imp.import_next(imp.java:838) at org.python.core.imp.import_module_level(imp.java:957) at org.python.core.imp.importName(imp.java:1057) at org.python.core.ImportFunction.__call__(__builtin__.java:1280) at org.python.core.PyObject.__call__(PyObject.java:450) at org.python.core.__builtin__.__import__(__builtin__.java:1232) at org.python.core.imp.importOne(imp.java:1076) at httplib$py.f$0(/opt/jython/Lib/httplib.py:1364) at httplib$py.call_function(/opt/jython/Lib/httplib.py) at org.python.core.PyTableCode.call(PyTableCode.java:171) at org.python.core.PyCode.call(PyCode.java:18) at org.python.core.imp.createFromCode(imp.java:436) at org.python.core.imp.createFromPyClass(imp.java:237) at org.python.core.imp.createFromPyClass(imp.java:205) at org.python.core.imp.loadFromSource(imp.java:657) at org.python.core.imp.find_module(imp.java:551) at org.python.core.imp.import_next(imp.java:838) at org.python.core.imp.import_module_level(imp.java:957) at org.python.core.imp.importName(imp.java:1057) at org.python.core.ImportFunction.__call__(__builtin__.java:1280) at org.python.core.PyObject.__call__(PyObject.java:450) at org.python.core.__builtin__.__import__(__builtin__.java:1232) at org.python.core.imp.importOne(imp.java:1076) at org.python.pycode._pyx282.f$0(scim_protected_policy.py:21) at org.python.pycode._pyx282.call_function(scim_protected_policy.py) at org.python.core.PyTableCode.call(PyTableCode.java:171) at org.python.core.PyCode.call(PyCode.java:18) at org.python.core.Py.runCode(Py.java:1614) at org.python.util.PythonInterpreter.execfile(PythonInterpreter.java:296) at org.gluu.service.PythonService.loadPythonScript(PythonService.java:181) ... 30 more Caused by: java.lang.OutOfMemoryError: Metaspace ==================Further details============================ Failed to load python file 'java.io.ByteArrayInputStream@2afda5fe' ``` Thank you.

Gluu 4.1.1 Ubuntu 18.04

closed

By Mohib Zico Account Admin 06 Jul 2021 at 9:43 a.m. CDT

Copy

Cloud Native support is not actually covered in community support subscription but I'll try to talk to our CN team to share some ideas.

By Mohammad Abudayyeh staff 06 Jul 2021 at 9:54 a.m. CDT

Copy

Hi Tiago, The maximum heap size as a percentage of total memory is set by default to 75.0 Which means you can increase the memory for the pod and it will increase the total memory available for the app. The env that controls that in the app is `GLUU_MAX_RAM_PERCENTAGE`/ Please contact our sales team if you wish to further engage support on Gluu cloud native. Thanks, Mohammad

By Tiago Fernandes user 06 Jul 2021 at 10:14 a.m. CDT

Copy

Hi Mohammad. First of all, thanks for the reply. My main question is: is this error, after the scripts update, related to this parameter? Will increasing this value eliminate this error? We have 8 or 9 custom UMA scripts deployed. Can you provide a suggested value for the parameter? Thank you, Tiago

By Mohammad Abudayyeh staff 06 Jul 2021 at 10:18 a.m. CDT

Copy

Tiago, All you need to do is increase memory for the pod that will automatically give java more mem. please engage our sales team for further support as we cannot engage further. Thank you, Mohammad

You need to Login in order to post an answer

Memspace out of memory after updating custom UMA policy

By: Tiago Fernandes user 28 Jun 2021 at 11:44 a.m. CDT

Answers

By Mohib Zico Account Admin 06 Jul 2021 at 9:43 a.m. CDT

By Mohammad Abudayyeh staff 06 Jul 2021 at 9:54 a.m. CDT

By Tiago Fernandes user 06 Jul 2021 at 10:14 a.m. CDT

By Mohammad Abudayyeh staff 06 Jul 2021 at 10:18 a.m. CDT

Post an answer