Skip to content

Commit

Permalink
Closes #1518: Make it possible to increase the amount of memory assig…
Browse files Browse the repository at this point in the history
…ned to the plaintext importer job

Introducing `plaintext_content_retriever_java_opts` input parameter set to `-Xmx2g` value by default.
  • Loading branch information
marekhorst committed Jan 10, 2025
1 parent a6d8524 commit c5eeb7c
Showing 1 changed file with 8 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@
<value>100000</value>
<description>maximum input data split size, required by streaming version reading DocumentContentUrl to split input data into more chunks</description>
</property>
<property>
<name>plaintext_content_retriever_java_opts</name>
<value>-Xmx2g</value>
</property>
<!-- S3 related -->
<property>
<name>objectstore_s3_keystore_location</name>
Expand Down Expand Up @@ -93,6 +97,10 @@
<delete path="${nameNode}${output}" />
</prepare>
<configuration>
<property>
<name>mapreduce.map.java.opts</name>
<value>${plaintext_content_retriever_java_opts}</value>
</property>
<!-- S3 content retrieval related -->
<property>
<name>hadoop.security.credential.provider.path</name>
Expand Down

0 comments on commit c5eeb7c

Please sign in to comment.