ODBC and DB2 - problem saving a large chunk of text - CWBNL0107
I'm using PyODBC to connect to DB2 and was seeing some problems saving certain rows of data. The column the error message mentioned was a CLOB type, and the error happened when the column had data larger than, say 10,000 characters. The error message reads thusly:
(pyodbc.DataError) ('22018', '[22018] [IBM][System i Access ODBC Driver]Column 6: CWBNL0107 - Converted 9739 bytes, 4869 errors found beginning at offset 0 (scp=1202 tcp=37 siso=1 pad=0 sl=9739 tl=19478) (30200) (SQLPutData); [22018] [IBM][System i Access ODBC Driver]Error in assignment. (30019)')
Googling the error, some people had mentioned the charset, so I tried tweaking that a little bit. Didn't work. Plus, some records were being saved, so it didn't make sense.
Then IBM themselves recommended turning on the Allow unsupported character option (in this case, through the AllowUnsupportedChar
/ALLOWUNSCHAR
arg in the connection string, though that part isn't really documented, like pretty much everything related to db2). I tried that, and I didn't get an error. But it mangled like half the data in a large row.
Parentheticals aside - not a good solution. I prefer the error, thank you very much. And I still suspected that the problem had to do with the size of the data.
Then I found this site which seems to be documenting all of the db2 connection string args, and it is a magical wonderland. I did a ctrl + f and searched for "length" and boom:
The MAXFIELDLEN keyword, (can also be specified as MaxFieldLength), controls how much LOB (large object) data is sent in a result set. The value indicates the size threshold in kilobytes and the default value is 15360 and in V5R2 the maximum value allowed is 2097152, (2MB). If a LOB is larger than this value, you will have to use subsequent calls to retrieve the rest of the LOB data in your application.
Now I think there's something wrong with the math there, because 2097152KB certainly isn't 2MB. The default was also far from 15360KB in my system. Anyways, I set MAXFIELDLEN=2056
in my connection string (see here for my post about the connection string) and everything worked like magico. I hope it works like magico for you, too.