Program proxy for Qiskit Runtime requests

[IceKhan13]

What is the expected behavior?

We need to have a sidecar that proxy all QiskitRutnime requests and associate runtime jobs to Program.

Benifits:

• associate primitives with programs
• when user kills program all associated primitives are also killed

Details

One way to implement this is to add list field primitives_executions to Job model and add primitives job ids to that list. Population of this field will be happening in sidecar (basically within sidecar we will be hitting api/v1/jobs/<job_id>/primitives endpoint).

When users run job.stop() from client we stop all associated primitives to that job.

Epic #1162

[akihikokuroda]

How can the primitive execution be hacked in the sidecar? This issue seems interesting but I'm not sure how to do it.

[IceKhan13]

Qiskit runtime is an Rest api service. It looks like client for runtime (QiskitRUntimeService) is using QISKIT_IBM_URL env variable to detect where to send requests.

https://github.com/Qiskit/qiskit-ibm-runtime/blob/83091ee488113f68a0fe101e435d9628131dca44/qiskit_ibm_runtime/accounts/management.py#L247

We can use this env variable to send request not to runtime, but to our sidecar which will be sending requests to runtime service and preserve ids of executed services.

For example inside of sidecar container we will have simple http service, something like this pseudocode:

@route("/")
def handle_runtime_request(request):
    # send request to runtime
    runtime_service_response = requests.post(url="RUNTIME_URL", data=request.data)

    # get primitive job id and save it to job 
    primitive_job_id = runtime_service_response.get("job_id")
    gateway_response = requests.post(
        "api/v1/jobs/<program_job_id>/primitives", 
        data={"primitive_id": primitive_job_id})

    return runtime_service_response

we still need to think how to pass program_job_id 🤔

[IceKhan13]

probably this issue can be splitted into 2:

• gateway: add associated primitives ids to job
• implement sidecar service

[akihikokuroda]

we may use tracestate for program_job_id propagation.

[akihikokuroda]

I wonder if this feature should be in the middleware. Should Qiskit or providers be responsible to the association between the program and primitives?

[IceKhan13]

I would say it should be in middleware, since we build on top of primitives. Primitives services does not know anything about higher level abstractions. At least it will be like that for now. Later on things might change :)

[akihikokuroda]

I see. Is there any other way to do this other than proxying the request in the sidecar? I wonder if it only works with the ibm_runtime.

[IceKhan13]

🤔 I was thinking about sidecar because it is easy to turn off, it's loosely coupled. But we are more than open for other suggestions :)

[IceKhan13]

we can proxy it within gateway itself :)
We will just add new endpoint and that is it

[akihikokuroda]

I see. I wonder if it's ibm_runtime specific implementation. As far as it works with generic providers, it's OK for me.

[akihikokuroda]

The job.id of the quantum-middleware can be in the environment variable at the primitive.run() is called.

[IceKhan13]

I see. I wonder if it's ibm_runtime specific implementation. As far as it works with generic providers, it's OK for me.

I think it is runtime specific.

The job.id of the quantum-middleware can be in the environment variable at the primitive.run() is called.

QiskitRuntimeService class has one of it's env variables called QISKIT_IBM_URL or something like that, which points to runtime service. We can swap it with our proxy url, where we will be updating records in Job to store all runtime requests.

[akihikokuroda]

Somehow the job.id of the quantum-middleware need to be in the runtime request.(probably in the job_tags. It can be in the request data) The job id of the runtime is in the response job_id=response["id"] for the runtime request.

[akihikokuroda]

For the keeping the primitive job ids, we can add a new table with a middleware job id and a primitive job id or we can add a field to the current Job model that has the primitive job id list in a string. I guess that the former is better. @IceKhan13 WDYT?

[IceKhan13]

Either way works fine.

If we want to go with second option, we will probably need to add new endpoint and handle writing this strings/list. It will be less code compared to option 1, I think 🤔

[psschwei]

A couple of general questions:

• How are we keeping the list of primitives in sync between the Job object and the database?
• Is the sidecar more or less a way to run the primitives async (i.e. an event queue)? Or is it more to just proxy/preserve info being passed between programs and primitives?

[akihikokuroda]

What I'm doing is like

more to just proxy/preserve info being passed between programs and primitives

I have to looked at what are passed between them in more details.

[akihikokuroda]

Qiskit runtime is an Rest api service. It looks like client for runtime (QiskitRUntimeService) is using QISKIT_IBM_URL env variable to detect where to send requests.

@IceKhan13 I don't see this env variable defined in the container running the program. It may not be simple to proxy the runtime request from the container.

[akihikokuroda]

@IceKhan13 I wonder if it is possible proxying the primitive requests without ibm_runtime_service change.

[akihikokuroda]

I found QiskitRuntimeService constructor taking proxy parameter. We may use it.

proxies (Optional[dict]) – Proxy configuration. Supported optional keys are urls (a dictionary mapping protocol or protocol and host to the URL of the proxy, documented at https://docs.python-requests.org/en/latest/api/#requests.Session.proxies (opens in a new tab)), username_ntlm, password_ntlm (username and password to enable NTLM user authentication)

[Tansito]

Reviewing the issue and the comments and seeing that we are going to need to start configuring Istio I was thinking if we could use it as a way to analyse the traffic too for some specific DNS's, in this case runtime:

https://istiobyexample.dev/monitoring-egress-traffic/

( Just come to my mind don't take it too seriously 😅 )

[akihikokuroda]

I'm not sure how we do this yet. Can we create a session (implicitly or explicitly by user option) so that we can cancel all jobs in the session?

[Tansito]

We didn't discuss it I think, Aki. Maybe we can save some time together and brainstorm ideas.

[akihikokuroda]

I put a sidecar container in the Ray node. I see the ray container is sending requests to 104.18.29.94. I guess the address is for the backend. I can not see this address in the Ray container environment variables or the ibmruntime service object or "ibmq_qasm_simulator" backend object so far. I'll check the contents of the request next.

[Tansito]

Thank you @akihikokuroda ! ❤️

Friday / Monday I will start working on this too and let you know advances

[akihikokuroda]

104.18.29.94 is api.quantum.ibm.com

Data sent for estimator.run are these.

{"program_id": "estimator", "params": {"circuits": [{"__type__": "QuantumCircuit", "__value__": "eJwL9Az29gzhYtBjgALGgkIGljQGDiCTlQEBmEBSULZ4oKO/I0yiuraQEayWkbGQARUwIukFAWYozQJXwQYVZWTADkKN3RNLUsHmpkGFOCR0XUJ+K/60z4QJwBSjqeB0IMF4RnKMZ4cazcSAHQRFRcGdD7YkFSqRB6Vh7jLwLc3RcMvJTyzRUDfSM1DXUSgoSk3OLM7Mz7M1NdbUUQiuzE3Kz9FQP7c52iBWXVOzrAxuCcxyptvPY5foPPZOXpl6VvbYNob9MAXnNoPDgRFPOARFwMOBciduwu3E1dNYj+++4pA6/XLsavuZGx3gTtxEQlQx0SOqmAZ/VFHBibSOKmZ6RBXz4I8qKjiR1lHFQo+oYhn8UcUwUFE1IJWJIZFheQBmBzAsB6QopYdDqVKQkONQOlfPOJ2IljphTTxGaEFCx2qJtk6kSnFMWydSoaSkTzFElSKdtpmGKkX6EIhucpzIwPAfCcB0AgDkvGDe"}], "observables": [{"__type__": "settings", "__module__": "qiskit.quantum_info.operators.symplectic.sparse_pauli_op", "__class__": "SparsePauliOp", "__value__": {"data": {"__type__": "settings", "__module__": "qiskit.quantum_info.operators.symplectic.pauli_list", "__class__": "PauliList", "__value__": {"data": ["IIIZZ", "IIZIZ", "IZIIZ", "ZIIIZ"]}}, "coeffs": {"__type__": "ndarray", "__value__": "eJyb7BfqGxDJyFDGUK2eklqcXKRupaBuk2xopq6joJ6WX1RSlJgXn1+UkgqScEvMKU4FihdnJBakAvkaJjqaOgq1CuQDLgYw+GDPgAKI5wMAzRsgJw=="}}}], "parameters": [[{"__type__": "Parameter", "__value__": "AAWrlgXHu9RAZZfTXas/mbFAzrJbMF0="}, {"__type__": "Parameter", "__value__": "AAWrlgXHu9RAZZfTXas/mbFBzrJbMV0="}, {"__type__": "Parameter", "__value__": "AAXb512kLONLY6llzR3GtgC/zrNbMF0="}, {"__type__": "Parameter", "__value__": "AAXb512kLONLY6llzR3GtgDAzrNbMV0="}]], "parameter_values": [[2.5628604836692714, 2.823762961794528, 5.937623207582237, 0.6271066160254924]], "transpilation_settings": {"skip_transpilation": false, "optimization_settings": {"level": 3}, "coupling_map": null, "basis_gates": null}, "resilience_settings": {"level": 0}, "run_options": {"shots": 4000, "init_qubits": true, "noise_model": null, "seed_simulator": null}}, "log_level": "WARNING", "backend": "ibmq_qasm_simulator", "start_session": true, "session_time": null, "hub": "ibm-q", "group": "open", "project": "main"}

{"program_id": "estimator", "params": {"circuits": [{"__type__": "QuantumCircuit", "__value__": "eJwL9Az29gzhYtBjgALGgkIGljQGDiCTlQEBmEBSULZ4oKO/I0yiuraQEayWkbGQARUwIukFAWYozQJXwQYVZWTADkKN3RNLUsHmpkGFOCR0XUJ+K/60z4QJwBSjqeB0IMF4RnKMZ4cazcSAHQRFRcGdD7YkFSqRB6Vh7jLwLc3RcMvJTyzRUDfSM1DXUSgoSk3OLM7Mz7M1NdbUUQiuzE3Kz9FQP7c52iBWXVOzrAxuCcxyptvPY5foPPZOXpl6VvbYNob9MAXnNoPDgRFPOARFwMOBciduwu3E1dNYj+++4pA6/XLsavuZGx3gTtxEQlQx0SOqmAZ/VFHBibSOKmZ6RBXz4I8qKjiR1lHFQo+oYhn8UcUwUFE1IJWJIZFheQBmBzAsB6QopYdDqVKQkONQOlfPOJ2IljphTTxGaEFCx2qJtk6kSnFMWydSoaSkTzFElSKdtpmGKkX6EIhucpzIwPAfCcB0AgDkvGDe"}], "observables": [{"__type__": "settings", "__module__": "qiskit.quantum_info.operators.symplectic.sparse_pauli_op", "__class__": "SparsePauliOp", "__value__": {"data": {"__type__": "settings", "__module__": "qiskit.quantum_info.operators.symplectic.pauli_list", "__class__": "PauliList", "__value__": {"data": ["IIIZZ", "IIZIZ", "IZIIZ", "ZIIIZ"]}}, "coeffs": {"__type__": "ndarray", "__value__": "eJyb7BfqGxDJyFDGUK2eklqcXKRupaBuk2xopq6joJ6WX1RSlJgXn1+UkgqScEvMKU4FihdnJBakAvkaJjqaOgq1CuQDLgYw+GDPgAKI5wMAzRsgJw=="}}}], "parameters": [[{"__type__": "Parameter", "__value__": "AAWrlgXHu9RAZZfTXas/mbFAzrJbMF0="}, {"__type__": "Parameter", "__value__": "AAWrlgXHu9RAZZfTXas/mbFBzrJbMV0="}, {"__type__": "Parameter", "__value__": "AAXb512kLONLY6llzR3GtgC/zrNbMF0="}, {"__type__": "Parameter", "__value__": "AAXb512kLONLY6llzR3GtgDAzrNbMV0="}]], "parameter_values": [[3.5628604836692714, 2.823762961794528, 5.937623207582237, 0.6271066160254924]], "transpilation_settings": {"skip_transpilation": false, "optimization_settings": {"level": 3}, "coupling_map": null, "basis_gates": null}, "resilience_settings": {"level": 0}, "run_options": {"shots": 4000, "init_qubits": true, "noise_model": null, "seed_simulator": null}}, "log_level": "WARNING", "backend": "ibmq_qasm_simulator", "session_id": "cn7ourqea9rm8bapv4q0", "hub": "ibm-q", "group": "open", "project": "main"}

From the response, job_id is captured and RuntimeJob object is created and returned i qiskit_runtime_service.py.

        job = RuntimeJob(
            backend=backend,
            api_client=self._api_client,
            client_params=self._client_params,
            job_id=response["id"],
            program_id=program_id,
            user_callback=callback,
            result_decoder=result_decoder,
            image=qrt_options.image,
            service=self,
        )

[Tansito]

@akihikokuroda how do you see to create a draft pull request and comment it together next week with mine? (I will try to create it monday / tuesday)

[akihikokuroda]

@Tansito So far, I'm still investigating what and how can be done in the sidecar. So I don't have anything to put in the PR except comments.

[Tansito]

Got it! In my case I started to do some mock-ups with Scapy and its sniff functionality (it uses tcdump under the hood). Looks good but same as you, I don't have anything integrated yet 😄

[akihikokuroda]

I wonder if the simple sidecar works. The communication between the client and the server is encrypted. The sidecar can not see the contents. It probably must be a TLS proxy to see the contents.

[akihikokuroda]

Scapy is under GPL-2.0. It may not be good for us to use.

[Tansito]

I wonder if the simple sidecar works. The communication between the client and the server is encrypted.

Mmm... Good catch... I don't think we can address that... Any idea? Because taking this into account I only see as solution an integration with the Runtime client (returning to other options like the callback one, and so on) 🤔

Scapy is under GPL-2.0. It may not be good for us to use.

Oh! Good catch, Aki! Thank you.

[akihikokuroda]

The runtime job id can be extracted in the proxy now. How can the proxy know the associated middleware job id for the runtime job id? I guess that the middleware job id must be put into the request header of the runtime job request. Is there any idea? @IceKhan13 @Tansito @psschwei

[Tansito]

I don't know any "standard" way to do this so probably a custom header would be my option to go.

[akihikokuroda]

[akihikokuroda]

@psschwei @IceKhan13 @Tansito☝️

[akihikokuroda]

To make this work, the QiskitRuntimeService must be created with verify=False. If Istio takes care of (m-)TLS among pods and each app doesn't use TLS, this proxy works (with some modifications), too.

[akihikokuroda]

Webserver based proxy is working. I can take out the iptables configuration when we get the way to fake the backend address.

[Tansito]

Do you think we can close this pull request, @akihikokuroda ?

[akihikokuroda]

Yes,we can. I'll open one to update the job.stop()tomorrow.