Skip to content
This repository has been archived by the owner on Oct 30, 2023. It is now read-only.

Commit

Permalink
This diff contains the following fixes:
Browse files Browse the repository at this point in the history
* Once the OOC is at OFFLAODING state, it may offload data to disk indefinitely. We should call GC manually and get OOC out of OFFLOADING if anything has changed in computation/communication pattern.
Fixes a "if" statement which entirely disabled regression for memory estimation.
* Interleaving of in resetting and calculating memory estimation potentially can cause data race.
* The superstep count in memory estimator was not coherent throughout the calculation.
* Sometime the memory estimator's accuracy is not good. We should fall back to a threshold-based scheme relying on a more pessimistic memory usage report (such as the one given by JVM – memory usage includes garbage data too, but it can be used as a pessimistic estimate, as it is currently used in ThresholdBasedOracle).

More tuning is needed for a smooth memory estimation mechanism.
"mvn clean install" passes all tests and checks.
  • Loading branch information
heslami committed Jun 16, 2017
1 parent ea7753f commit a7ab3ce
Show file tree
Hide file tree
Showing 2 changed files with 228 additions and 102 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -426,11 +426,14 @@ public void ioCommandCompleted(IOCommand command) {
* threads.
*
* @param fraction the fraction of processing threads to remain active. This
* number is in range [0, 1]
* number is in range (0, 1]
*/
public void updateActiveThreadsFraction(double fraction) {
checkState(fraction >= 0 && fraction <= 1);
int numActiveThreads = (int) (numProcessingThreads * fraction);
// Making sure the number of active threads is not set to 0 to ensure
// progress in computation
int numActiveThreads = Math.max(
(int) (Math.ceil(numProcessingThreads * fraction)), 1);
if (LOG.isInfoEnabled()) {
LOG.info("updateActiveThreadsFraction: updating the number of active " +
"threads to " + numActiveThreads);
Expand Down Expand Up @@ -489,7 +492,7 @@ public void processingThreadFinish() {
*/
public void updateRequestsCreditFraction(double fraction) {
checkState(fraction >= 0 && fraction <= 1);
short newCredit = (short) (maxRequestsCredit * fraction);
short newCredit = (short) (Math.ceil(maxRequestsCredit * fraction));
if (LOG.isInfoEnabled()) {
LOG.info("updateRequestsCreditFraction: updating the credit to " +
newCredit);
Expand Down
Loading

0 comments on commit a7ab3ce

Please sign in to comment.