An interesting tangential thing that occurred to me while thinking more about the file-backed TVB issues. If we make all TVBs lazy (fetching data only when needed) and add proper reference-counting for them (instead of the sort-of-ref-counting that the chaining provides) then you could still build reassembly lazily with decrypted packets, by storing callbacks for doing TVB transforms. Eg:
Packet A is a composite of packets B and C.
Packet B is just the nth packet in the capture file.
Packet C is the result of passing packet D through an decryption callback.
Packet D is the result of passing packet E through a decompression callback.
...
Packet Z is just the nth packet in the capture file.
These can be chained forever without much memory cost. Obviously the cost to rebuild a TVB gets higher (and it would require a huge API change) but most of the time TVBs are rebuilt is when selecting a single packet (so it's still generally cheap) or when scanning the file to filter etc. in which case proper caching should alleviate most of the cost.
Cheers,
Evan
