fq lint fails on large files

[bounlu]

Hi,

It seems fq lint fails on large FASTQ file due to below error:

fq lint XXX_1.merged.fastq.gz XXX_2.merged.fastq.gz > XXX.fq_lint.log.txt

Command error:
  Error: I/O error
  Caused by:
      corrupt deflate stream

The file size is more than 50GB per file.

[zaeleus]

Can you ensure your inputs are indeed not corrupt? I.e., gzip --verbose --test XXX_1.merged.fastq.gz XXX_2.merged.fastq.gz.

[apeltzer]

Yeah cannot really confirm this. I have run it on inputs as large as 75GB per file (PE mode) and it ran fine 🤷🏻

[bounlu]

I have regenerated the gzipped files and tested with fq lint. The files were previously corrupted as you indicated as they passed fq lint this time.