CSV: Use escapeChar in nullValue

[tabalinas]

I have an issue with CSV reader, when escapeChar is set to \ and nullValue to \N.

In this case reading a CSV like the following:

id,value
1,\N

results in value="N" instead of value=null.

I tried to replace nullValue with "\\\\N", but it doesn't help.

Any ideas?

Thank you in advance!

[cowtowncoder]

I think a unit test (piece of code) that reproduces the issue would help here, to ensure specifics of how characters are escaped in code.

Also: which Jackson version is this with?

[tabalinas]

Thank you for the quick response!

Sure, a unit test to reproduce the issue:

@Test
public void escapeCharAndNullValueConflict() throws IOException {
    String csv = "id,value\n" +
            "1,\\N";

    InputStream in = new ByteArrayInputStream(csv.getBytes(StandardCharsets.UTF_8.name()));

    CsvSchema schema = CsvSchema
            .emptySchema()
            .withHeader()
            .withColumnSeparator(',')
            .withEscapeChar('\\')
            .withNullValue("\\N");

    MappingIterator<ObjectNode> it =
            new CsvMapper().readerFor(ObjectNode.class).with(schema).readValues(in);

    ObjectNode row = it.nextValue();

    assertThat(row.get("value"), is(nullValue()));
}

which fails with:

Expected: is null
     but: was <"N">
java.lang.AssertionError: 
Expected: is null
     but: was <"N">
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at org.junit.Assert.assertThat(Assert.java:865)
	at org.junit.Assert.assertThat(Assert.java:832)

We use Jackson v2.8.8.

[cowtowncoder]

Hmmh. I am not 100% sure what the logic should be, and due to the way tokenization is done it may be that this is considered feature, not bug. The problem is that replacement of null value is higher level operation and occurs after tokenization; escape characters are, however, handled during tokenization.

But I can see why this may be problematic so I will keep this open to see if handling could be improved.