You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently we dont have a return channel for info, warning and debugging info from inside the boefjes. And, with the containerized option we have no other route than either std-out, or std-err. Std-err currently signals that all output is either a stacktrace or at least something unstructured, (so can only contain 1 datablob), and std-out is always structured to contain a set of mime-type(s)+rawfile combinations.
It would fairly simple to specify a few 'special case' mime-types in that structured std-out to hold various levels of debugging output.
Describe the solution you'd like
I propose we standardize a couple of new mime-types next to error/boefje:
debug/boefje
warning/boefje
info/boefje
These could then be used as a side-channel to get data out of the boefje that is not meant for parsing by the usual normalizers, but would provide more context on what happened inside the boefje.
A direct usecase for this would be the shodan boefje which currently silently ignores local ip's. the boefje then completes successfully, without any user visible output. An error for this boefje might also be reasonable I' guess, but that would be indicative of something you'd need to fix.
The scheduler currently special cases raw files that has a mimetype starting with error/ this would for now need to be extended to also exclude the above mimetypes.
Describe alternatives you've considered
No other options come to mind. Other mime-types could be considered, but these seem to be inline with the current error mime-type, and it would be easier to keep them separate when (in the future) we can target boefje/* or info/* as a consumable mime-type for a normalizer.
Additional context
Depending on some security questions, we might even consider showing the contents of some of these mime-types directly in side user-interface. (as plain text). (see https://docs.openkat.nl/guidelines/ideas.html#safe-viewing-of-boefjes-data-dutch)
There is some issue with boefjes describing their own mime-types, as this would open the door to (for example) the download boefje being tricked into 'downloading' a file from the web with one of these mime-types, and then returning it as is.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Currently we dont have a return channel for info, warning and debugging info from inside the boefjes. And, with the containerized option we have no other route than either std-out, or std-err. Std-err currently signals that all output is either a stacktrace or at least something unstructured, (so can only contain 1 datablob), and std-out is always structured to contain a set of mime-type(s)+rawfile combinations.
It would fairly simple to specify a few 'special case' mime-types in that structured std-out to hold various levels of debugging output.
Describe the solution you'd like
I propose we standardize a couple of new mime-types next to error/boefje:
These could then be used as a side-channel to get data out of the boefje that is not meant for parsing by the usual normalizers, but would provide more context on what happened inside the boefje.
A direct usecase for this would be the shodan boefje which currently silently ignores local ip's. the boefje then completes successfully, without any user visible output. An error for this boefje might also be reasonable I' guess, but that would be indicative of something you'd need to fix.
The scheduler currently special cases raw files that has a mimetype starting with
error/
this would for now need to be extended to also exclude the above mimetypes.Describe alternatives you've considered
No other options come to mind. Other mime-types could be considered, but these seem to be inline with the current error mime-type, and it would be easier to keep them separate when (in the future) we can target boefje/* or info/* as a consumable mime-type for a normalizer.
Additional context
Depending on some security questions, we might even consider showing the contents of some of these mime-types directly in side user-interface. (as plain text). (see https://docs.openkat.nl/guidelines/ideas.html#safe-viewing-of-boefjes-data-dutch)
There is some issue with boefjes describing their own mime-types, as this would open the door to (for example) the download boefje being tricked into 'downloading' a file from the web with one of these mime-types, and then returning it as is.
The text was updated successfully, but these errors were encountered: