Define systems #2034
Comments
|
List of Collapsed for brevityAs found by |
|
So we need to define what is a single system. |
|
I think we defined a system as: Multiple hostnames that resolve to one ip address where at least one of the hostnames or the ip address has a declared scan level that is at least L1. What the system type is should be defined by what ip services are served by the machine with that ip address. Since we cannot see whether the ipv4 and ipv6 addresses are addresses of a single, or multiple machines, we define then as multiple systems. |
|
The declared scan level part does not have to queried since that query is done at the report ooi selection step. |
There is indeed no definite way, but practically we could compare fingerprints; especially when there is encryption involved. |
|
Yes for sure. I meant with the current OOIs, should've been more specific. |
|
Looking at this again, should we consider it closed @noamblitz @underdarknl? |
This might not be enough, we might also need to be able to look at what software is running. |
|
So I've been thinking about it and think we really first should discuss it with stakeholders. If we are going to keep systems defined like they are now (purely service-based), a simple BIT should be the solution. If there is more, such as Software like @underdarknl is mentioning, it will quickly become too complicated for the current BITS. For example, Software -> SoftwareInstance -> IPService -> IPPort -> IPaddress is not possible in the BITS. We should also keep in mind that there is no software discovery on ipservices currently, so including that in the system definition will force us to write boefjes for that (delaying a bit). |
Rieven
changed the title
|
What is the problem that we are trying to solve? This is not entirely clear to me. Is that the very 'basic' definition of a system? Is it that the naming of the 'systems report' is confusing? Something else? |
|
I made an 'exercise' sheet so the community can help us out. Any suggestions, comments or things that could be improved?
|
|
Input from discussion meeting:
Currently systems work with IPServices, so when a hostname has an IP and the IP offers HTTPS, then we say that IP+Hostname is a system, with the system type 'Web'. This only works for situations where IPservices are detected. When there is no IPService, there is currently also no 'system'. |
|
You can also have the following scenario: thousands of hostnames with different owners that point the same IP address (for example cloudflare) |
|
Conclusion discussionmeeting 23/04/2024:
|
|
Adjusted image with some added services (HTTP, SSH, SMTP):
|
|
Another suggestion could be to add tags to IPs and/or hostnames such that the user can define whether or not the IPs are part of a cloud solution and/or shared hosting, as these are the issues where issues would occur. Based on these tags we can then 'change' the definition of a system as we go. We could perform some basic checks ourselves by performing whois queries on the IP space and combine this with perhaps the DNS configuration for various cloud solutions. If a cloud solution like AWS/Azure etc is detected we can automatically give it this tag. A suggestion from @underdarknl was to do this using bits that read a config. |
|
The definition of a system remains complex. Everybody had issues filling in the 'define systems' templates from above. During the discussion meeting we defined the problems we are facing, and plan to create separate solutions for each of the problems, instead of solving the 'one big problem'. These current identified problems are:
Additional discussion: |
Part 1:
From the mock report
This is fixed by:
Part 2:
What is a single system?
The definition of a system is now implicitly defined in the systems report.
Important
There exists some ambiguity in the code with naming systems and services. One system can provide multiple services, such as Web, Mail, and Dicom. We currently call the latter the system type. Hence a system that is a Web System can also be a Mail System.
Current situation
The current implementation aggregates the following information from Octopoes to create a system:
Information used in a System
Example of resulting definition of a System
Current issues
Possible solutions
There are a few things to fix:
The latter could be tackled in two ways.
1. Introducing objects to pre-annotate a lot of steps
We could leverage Bits to infer the system types from the services and software, which is what the Website object in fact is. This would mean bits such as website_discovery that take in a ResolvedHostname and IPService to add objects such as NameServer and MailServer. Here the issue might be that we do not want to depend on the hostname, so these could also only take in an IPService.
Furthermore, we could consider a new "System" object that aggregates Websites, MailServers and NameServers. Or perhaps just aggregate ips and query the Websites etc.?
Sketch if we introduce a System object (WIP, dashed line meaning optional)
2. Leverage current structure and make it more consistent
We are re-introducing the concept of a service here a bit, where we now focus on the categorization of a service and do some additional queries to enhance the result set. For instance, we query the IPService object to let nmap find mail servers, but use the Website object to determine if we are dealing with a web service. If we update the
Servicemodel a bit by introducing atypefield.We can then introduce bits that infer the service type from a Service, and write bits that add (IP) Services for the edge cases for Websites and SoftwareInstances where these objects imply an ip service. UPDATE: looking at the
website_discoverybit, actually it seems that the same service-mapping logic is applied. So if we want more consistency here, we would redefine a Website as the result of querying objects in this path with a filter on theService.type:And the Mail, Name and other services would be defined accordingly. A bit would infer the system types.
We could then still add a definition of a system by either aggregating ip addresses or ip services.
The text was updated successfully, but these errors were encountered: