After cwltool error, re-submitting gets duplicate key error

[tetron]

Description

Submit a workflow with cwltool broken on the backend. It will fail. Then restart the server with a fixed cwltool and re-submit, instead of processing the file successfully, the request crashes.

Expected Behavior

It should process the file successfully on the second try.

Actual Behavior

2021-04-30 12:07:37.502 ERROR 3819 --- [cTaskExecutor-2] org.commonwl.view.cwl.CWLToolRunner      : Unexpected error

org.springframework.dao.DuplicateKeyException: Write failed with error code 11000 and error message 'E11000 duplicate key error collection: test.workflow index: retrievedFrom dup key: { : { repoUrl: "https://github.com/common-workflow-language/workflows.git", branch: "lobstr-v1", path: "workflows/lobSTR/lobSTR-workflow.cwl" } }'; nested exception is com.mongodb.DuplicateKeyException: Write failed with error code 11000 and error message 'E11000 duplicate key error collection: test.workflow index: retrievedFrom dup key: { : { repoUrl: "https://github.com/common-workflow-language/workflows.git", branch: "lobstr-v1", path: "workflows/lobSTR/lobSTR-workflow.cwl" } }'
	at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:73) ~[spring-data-mongodb-1.10.23.RELEASE.jar:na]
	at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2128) ~[spring-data-mongodb-1.10.23.RELEASE.jar:na]
	at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:461) ~[spring-data-mongodb-1.10.23.RELEASE.jar:na]
	at org.springframework.data.mongodb.core.MongoTemplate.insertDBObject(MongoTemplate.java:1026) ~[spring-data-mongodb-1.10.23.RELEASE.jar:na]
	at org.springframework.data.mongodb.core.MongoTemplate.doInsert(MongoTemplate.java:835) ~[spring-data-mongodb-1.10.23.RELEASE.jar:na]
	at org.springframework.data.mongodb.core.MongoTemplate.insert(MongoTemplate.java:776) ~[spring-data-mongodb-1.10.23.RELEASE.jar:na]
	at org.springframework.data.mongodb.repository.support.SimpleMongoRepository.save(SimpleMongoRepository.java:80) ~[spring-data-mongodb-1.10.23.RELEASE.jar:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.executeMethodOn(RepositoryFactorySupport.java:515) ~[spring-data-commons-1.13.23.RELEASE.jar:na]
	at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:500) ~[spring-data-commons-1.13.23.RELEASE.jar:na]
	at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:477) ~[spring-data-commons-1.13.23.RELEASE.jar:na]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.25.RELEASE.jar:4.3.25.RELEASE]
	at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:56) ~[spring-data-commons-1.13.23.RELEASE.jar:na]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.25.RELEASE.jar:4.3.25.RELEASE]
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.3.25.RELEASE.jar:4.3.25.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.25.RELEASE.jar:4.3.25.RELEASE]
	at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:57) ~[spring-data-commons-1.13.23.RELEASE.jar:na]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.25.RELEASE.jar:4.3.25.RELEASE]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) ~[spring-aop-4.3.25.RELEASE.jar:4.3.25.RELEASE]
	at com.sun.proxy.$Proxy105.save(Unknown Source) ~[na:na]
	at org.commonwl.view.cwl.CWLToolRunner.createWorkflowFromQueued(CWLToolRunner.java:97) ~[classes/:na]
	at org.commonwl.view.cwl.CWLToolRunner$$FastClassBySpringCGLIB$$cd465805.invoke(<generated>) ~[classes/:na]
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.3.25.RELEASE.jar:4.3.25.RELEASE]
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) ~[spring-aop-4.3.25.RELEASE.jar:4.3.25.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.3.25.RELEASE.jar:4.3.25.RELEASE]
	at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:115) ~[spring-aop-4.3.25.RELEASE.jar:4.3.25.RELEASE]
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: com.mongodb.DuplicateKeyException: Write failed with error code 11000 and error message 'E11000 duplicate key error collection: test.workflow index: retrievedFrom dup key: { : { repoUrl: "https://github.com/common-workflow-language/workflows.git", branch: "lobstr-v1", path: "workflows/lobSTR/lobSTR-workflow.cwl" } }'
	at com.mongodb.operation.BaseWriteOperation.convertBulkWriteException(BaseWriteOperation.java:243) ~[mongodb-driver-core-3.4.3.jar:na]
	at com.mongodb.operation.BaseWriteOperation.access$300(BaseWriteOperation.java:60) ~[mongodb-driver-core-3.4.3.jar:na]
	at com.mongodb.operation.BaseWriteOperation$1.call(BaseWriteOperation.java:144) ~[mongodb-driver-core-3.4.3.jar:na]
	at com.mongodb.operation.BaseWriteOperation$1.call(BaseWriteOperation.java:133) ~[mongodb-driver-core-3.4.3.jar:na]
	at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:426) ~[mongodb-driver-core-3.4.3.jar:na]
	at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:417) ~[mongodb-driver-core-3.4.3.jar:na]
	at com.mongodb.operation.BaseWriteOperation.execute(BaseWriteOperation.java:133) ~[mongodb-driver-core-3.4.3.jar:na]
	at com.mongodb.operation.BaseWriteOperation.execute(BaseWriteOperation.java:60) ~[mongodb-driver-core-3.4.3.jar:na]
	at com.mongodb.Mongo.execute(Mongo.java:845) ~[mongodb-driver-3.4.3.jar:na]
	at com.mongodb.Mongo$2.execute(Mongo.java:828) ~[mongodb-driver-3.4.3.jar:na]
	at com.mongodb.DBCollection.executeWriteOperation(DBCollection.java:342) ~[mongodb-driver-3.4.3.jar:na]
	at com.mongodb.DBCollection.insert(DBCollection.java:337) ~[mongodb-driver-3.4.3.jar:na]
	at com.mongodb.DBCollection.insert(DBCollection.java:328) ~[mongodb-driver-3.4.3.jar:na]
	at com.mongodb.DBCollection.insert(DBCollection.java:298) ~[mongodb-driver-3.4.3.jar:na]
	at com.mongodb.DBCollection.insert(DBCollection.java:264) ~[mongodb-driver-3.4.3.jar:na]
	at com.mongodb.DBCollection.insert(DBCollection.java:201) ~[mongodb-driver-3.4.3.jar:na]
	at org.springframework.data.mongodb.core.MongoTemplate$9.doInCollection(MongoTemplate.java:1031) ~[spring-data-mongodb-1.10.23.RELEASE.jar:na]
	at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:459) ~[spring-data-mongodb-1.10.23.RELEASE.jar:na]
	... 28 common frames omitted

Possible Fix

It needs to correctly clean up after the first error.

Steps to Reproduce

1. Run mvn spring-boot:run without cwltool in the path
2. Submit a workflow. The request will fail.
3. Restart the service with cwltool in the path.
4. Submit the same workflow. The request will fail with a duplicate key error.

Context

In development, accidentally started the server without the cwltool virtualenv.

Your Environment

• Version used: 0a5b427