-
Notifications
You must be signed in to change notification settings - Fork 1
Home
Since degressly works as an HTTP Proxy, intercepting SSL requests takes some good ol' man in the middle interception.
We recommend using mitmproxy as the HTTPS_PROXY to intercept the https requests and forward it to degressly-downstream as an HTTP request. A sample config for mitmproxy follows:
from mitmproxy import http
def request(flow: http.HTTPFlow) -> None:
original_host = flow.request.host_header
original_scheme = flow.request.scheme
flow.request.host = 'degressly-downstream'
flow.request.port = 8080
flow.request.scheme = "http"
# Change the Host header to the original destination
if original_host:
flow.request.host_header = original_host
# Optionally, log or debug the redirected flow
print(f"Redirecting HTTPS request to HTTP: {flow.request.url}")
flow.request.headers["X-Forwarded-Proto"] = original_schemeYou will have to provide mitmproxy with a Root CA certificate or let mitmproxy generate one of its own, and that certificate then must be added to the trust store of your docker containers.
Refer to the mitmproxy certificate authority documentation for more details regarding certificate setup.
For example, the following directives can be added in Ubuntu/Debian based images for adding trusted certificates:
...
COPY ./certs/mitmproxy-ca-cert.crt /usr/local/share/ca-certificates/mitmproxy-ca-cert.crt
RUN update-ca-certificates
...As an addendum, some runtimes like python and JVM do not use the OS trusted root CAs for their trust store. In such cases, you may have to manually configure the certificates. For example, if you are using the python requests library you can add the following directive to your Dockerfile:
...
ENV REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
...A working example for easy reference is present in degressly/degressly-demo with the MITM interceptor.
The degressly-downstream service requires the presence of a x-degressly-caller header in each HTTP request to determine which replica is calling the given endpoint so that it can handle non-idempotent requests and also to determine the calling replica for making observations. If you do not wish to modify your code to inject the header into all our downstream requests, you may setup 3 mitmproxy instances with code to inject this header for each replica, and point each of your replicas to the respective mitmproxy instance.
To begin with, create an mimtproxy script that populates this header based on an environment variable:
from mitmproxy import http
import os
def request(flow: http.HTTPFlow):
original_host = flow.request.host_header
original_scheme = flow.request.scheme
flow.request.host = 'degressly-downstream'
flow.request.port = 8080
flow.request.scheme = "http"
# Change the Host header to the original destination
if original_host:
flow.request.host_header = original_host
# Optionally, log or debug the redirected flow
print(f"Redirecting HTTPS request to HTTP as: {flow.request.url}")
flow.request.headers["X-Forwarded-Proto"] = original_scheme
flow.request.headers["x-degressly-caller"] = os.getenv("PROXY_HEADER")Now, to create three separate instances of the mitm proxy as follows:
degressly-mitm-primary:
image: mitmproxy/mitmproxy
container_name: "degressly-mitm-primary"
environment:
PROXY_HEADER: PRIMARY
volumes:
- ./certs:/home/mitmproxy/.mitmproxy
- ./mitm.py:/mitm.py:ro
command: ["mitmdump", "-s", "mitm.py", "--no-http2"]
networks:
- degressly_demo_network
depends_on:
- degressly-downstream
degressly-mitm-secondary:
..
container_name: "degressly-mitm-secondary"
environment:
PROXY_HEADER: SECONDARY
..
degressly-mitm-candidate:
..
container_name: "degressly-mitm-candidate"
environment:
PROXY_HEADER: CANDIDATE
..Point each of your replicas to the appropriate proxy server
degressly-demo-primary:
..
environment:
HTTP_PROXY: https://degressly-mitm-primary:8080
HTTPS_PROXY: https://degressly-mitm-primary:8080
..
degressly-demo-secondary:
..
HTTP_PROXY: https://degressly-mitm-secondary:8080
HTTPS_PROXY: https://degressly-mitm-secondary:8080
..
degressly-demo-candidate:
..
HTTP_PROXY: https://degressly-mitm-candidate:8080
HTTPS_PROXY: https://degressly-mitm-candidate:8080
..Now, each instance will perform calls to upstream services via it's own proxy, which in turn will inject the degressly caller header for the respective replica of code.
A working example for easy reference is present in degressly/degressly-demo with the MITM interceptor.
degressly-downstream supports Groovy based configuration files for use cases such as:
- When your application calls a wide array of upstream services, some idempotent and some not.
- When changes cannot be made to the code for obtaining the idempotency keys or trace IDs of requests.
A groovy script that configuration has to implement the methods of com.degressly.proxy.downstream.handler.DownstreamHandler.
A sample Groovy Config is as follows:
package config
import com.degressly.proxy.downstream.dto.RequestContext
import groovy.json.JsonSlurper
import org.springframework.util.MultiValueMap
class DownstreamHandler implements com.degressly.proxy.downstream.handler.DownstreamHandler {
Set<String> idempotentURIs = Set.of("/sample-idempotent", );
@Override
Optional<Boolean> isIdempotent(RequestContext requestContext) {
if (idempotentURIs.contains(requestContext.getRequest().getRequestURI())) {
return Optional.of(Boolean.TRUE)
} else {
return Optional.of(Boolean.FALSE)
}
}
@Override
Optional<String> getTraceId(RequestContext requestContext) {
JsonSlurper jsonSlurper = new JsonSlurper()
def bodyJson
try {
bodyJson = jsonSlurper.parseText(requestContext.getBody())
} catch(Exception ignored) {
// Do nothing
bodyJson = null
}
def optional
optional = getField(requestContext.getHeaders(), requestContext.getParams(), bodyJson,"trace-id")
if (optional.isPresent())
return Optional.of(optional.get())
optional = getField(requestContext.getHeaders(), requestContext.getParams(), bodyJson,"seqNo")
if (optional.isPresent())
return Optional.of(optional.get())
optional = getField(requestContext.getHeaders(), requestContext.getParams(), bodyJson,"seq-no")
if (optional.isPresent())
return Optional.of((optional.get()))
optional = getField(requestContext.getHeaders(), requestContext.getParams(), bodyJson,"txn-id")
if (optional.isPresent())
return Optional.of((optional.get()))
optional = getField(requestContext.getHeaders(), requestContext.getParams(), bodyJson,"txnId")
if (optional.isPresent())
return Optional.of(optional.get())
return Optional.empty()
}
@Override
Optional<String> getIdempotencyKey(RequestContext requestContext) {
if (requestContext.getTraceId()) {
return Optional.of(requestContext.getRequest().getRequestURL().append("_").append(requestContext.getTraceId()).toString());
}
return Optional.empty();
}
private static Optional<String> getField(MultiValueMap<String, String> headers, MultiValueMap<String, String> params, Object bodyJson, String field) {
if (headers.containsKey(field))
return Optional.of(headers.getFirst(field))
if (params.containsKey(field))
return Optional.of(params.getFirst(field))
if (bodyJson!=null && bodyJson[field] != null && bodyJson[field] instanceof String)
return Optional.of((String) bodyJson[field])
return Optional.empty()
}
}This config does the following:
- declares the
/sample-idempotentAPI as idempotent. i.e., each replica's calls will be forwarded to the upstream system. Whereas for all other APIs will be treated as non-idempotent. - extracts the traceId based on whichever field is found first in the request as per the order of precedence in the
getTraceIdmethod. - returns _ as the idempotency key.
WIP