LongRange Developer Forum

Hi all.

Although I've tagged this as [RDMLX/RPG] my specific scenario relates to an RDMLX application, but I think the question is relevant to all.

So anyway, I've got an existing offline LR application which uploads data to the server. The upload comprises of a number of different types of data, sourced from a number of different SQLite tables on the device. Currently all the data goes up in one block and is processed in one go on the server. Because the different datasets relate to each other I initially dump the uploaded data into working lists from where I can manipulate and process the related data, eventually, once processed it is written to database files up on the IBMi.

This has worked fine for a few years - but due to application growth, network and server loading etc the uploads are now beginning to fail / timeout. So, what I want to do is use dbuploadsegmentsizelimit to split the upload into smaller chunks, to reduce the chance of a timeout, so the question is, does the executing server program stay resident in memory, with field values and list contents intact until all of the chunks have been uploaded... or is a clean start on the server for each chunk. If its the latter, and because all my data is connected and needs to be processed in one go, I think I'm going to have to create a bunch of intermediate tables on the IBMi to initially accumulate the uploaded data, and do my processing from that, rather than using working lists.

I just wondered if anybody has used dbuploadsegmentsizelimit for related multi-table uploads and has faced a similar situation. I'm going to run some tests and will post my findings here, but just thought I'd throw this out to the community.

Cheers for now.

Marcus.

Hi again

So I ran some tests and, as expected, as far as I can see field values / list contents are not preserved in the server side logic. So in complex scenarios with multiple tables being uploaded you would need to consider a strategy to accumulate the data so it could be processed in one go after the final block was uploaded. Probably the easiest way would be to use a file/table which would be retained beyond the scope of the executing logic. With each block of uploaded, data would be added to / appended to the file, and then it would be cleared down after the processing was complete - obviously to keep uploaded data from different devices separate you would need to ensure that the data was tagged uniquely. The device GUID;

#com_owner.Get_ClientGUID ClientGUID(#wkGUID)

could be used for that purpose (for example as a key column on a table.

Anyway - hope that info is useful.

Cheers for now.

Marcus.

Hi again.

I've learnt an important fact when using DbUploadSegmentSizeLimit which I felt I needed to share here.

The HasMoreData value;

#com_Owner.Get_Database Tablestartidx(#wkFirstTable) Tablecount(#wkTableCount) HasMoreData(#wkHasMoreData)

tells your server logic when you need to stop processing upload blocks - if there is still data to process (HasMoreData = TRUE) you should jump out of the server logic so that the client continues to send further blocks. However for each block uploaded, before you jump out of the server logic you must run this method and set the OK parameter to TRUE;

#com_Owner.Set_Database Databaseok(True)

to indicate that this block has been processed - failure to do this will result in misleading data returned here;

#com_Owner.Get_DatabaseTable Tablenumber(#wkCurrentTable) Batchrowcount(#wkBatchRowCount) Batchrowstartidx(#wkBatchFirstRow) Tablename(#wkCurrentTableName)

Incorrectly, BatchRowCount will hold a total of all rows uploaded in all blocks processed so far (not reflecting the number of rows in the current block) and BatchRowStartIdx will hold a value which always points to the first row in the first block (i.e it will always be 1). Running this;

#com_Owner.Set_Database Databaseok(True)

after each block ensures that the client only attempts to send unprocessed data to the server and therefore the BatchRowCount and BatchRowStartIdx values will relate to the current block only.

Anyway - hope this helps - and big thanks to the guys in LANSA LongRange support for helping me understand how this all works a little better!

Cheers for now.

Marcus.