Improve task-based processing

[joachimmetz]

Task-based processing changes (to do after 1.5.0 release):

• Added attribute container identifiers
  • remove the need for storage_number, storage_index and event_uuid
• remove event uuid
• Clean up after adding attribute container identifiers
• Migrate psort to task-based processing
• deprecate and remove SingleProcessQueue
• clean up PsortAnalysisReportZIPStorageFileWriter
• clean up event buffer, and different event heaps
  • removed event buffer https://codereview.appspot.com/329770043/
• Storage object change SetEnableProfiling to EnableProfiling and DisableProfiling
• Clean up ProcessSources method (Also see: https://codereview.appspot.com/301940043/diff/20001/plaso/frontend/extraction_frontend.py?context=10&column_width=80)
• move remainder of collector, filter file, to worker?
• Salvage http://codereview.appspot.com/303250043/
• improve performance
  • if there are no new event sources workers are idle. prioritize processing of directories first?
• Single processing investigate initial wait time (first call to ProcessPathSpec)
• investigate and fix newly spawned worker remaining idle after other working being killed

Stored information:

• move buffer size into storage metadata
• Write source path spec information (mount points?)
• store extraction and analysis information
• store source scanner/volume information
• store collection errors, extraction (parsing) errors, etc.
• change event object to use event source
• store contents of filter file in storage for provenance?

Unsorted:

• handle sigterm of main process as signal abort
• worker: look into replacing "file_entry = None" with explicit invocation of GC
• Look at (https://codereview.appspot.com/301940043/diff/40001/tests/multi_processing/multi_process.py?context=10&column_width=80)
• have limited number of tasks scheduled before merging?

[Onager]

@joachimmetz Can you please update this issue? I suspect many of the checkboxes are no longer relevant, or have been partially event.

If you want to close out this issue, and open new issues for the individual pieces, I'm very supportive.

[joachimmetz]

@Onager I've started clean this up a bit