Trouble Exporting Larger Data Sets

I am currently having some issues with exporting larger annotations from ImFusion Labels.

I recently tried to export an annotation only for Labels to crash.
For reference, when I typically export annotations, my program will buffer for about 30 seconds (give or take) before eventually coming back online and proceeding successfully with the exportation.
I’ve noticed that the larger the file I export, the longer the program will buffer before eventually succeeding. It would appear though that I have hit some sort of a limit.
I am trying to export a file with 2,500+ frames (though only one frame has actually been annotated) with the following settings:

Note that I have tried unchecking the Split multi-image box to no avail.
When I try exporting a smaller set of annotations (2,000 frames or lower), with the same settings, after buffering it WILL successfully export.

When exporting this larger file, the only thing that changes in the export folder is the ‘data_list’ and ‘export_options’ files, but no annotation file will be present.

Please let me know if there is anything I can do in order to export this annotation successfully!

Hi William,
what I can imagine given that you say the system is “buffering” for quite some time, is that you are running out of memory. Could you look into the ImFusion Labels log file (on my Windows machine C:\Users\YourUser\AppData\Roaming\ImFusion\ImFusion Labels\ImFusionLabels.log otherwise $HOME/.ImFusion/ImFusionLabels.log on Ubuntu) and see if anything in that direction is logged?
Best,
Federico

Hi Federico,

Here is a screen shot of the most recent activity recorded on the mentioned file after trying to export the file:

Is this of any assistance?

Hi William,
unfortunately it is not immediately clear from the log what the problem might be.
Could you try monitoring your memory consumption when trying to export the dataset?
If memory consumption is very high, this might be due to an issue with previous versions of Labels (<0.22.3). To see if this is actually the case, you can try selecting and loading another (small) dataset before exporting (so that the large one is unloaded from memory). Then try exporting again. If export succeeds then this is due to an issue with ImFusionLabel’s version < 0.22.3: I would then upgrade the software if possible.
Best,
Federico

Hi Federico,

I am currently using version 3.10.3.

I’ve tried both freeing up some disc space and exporting a smaller file before the larger one, however, this sadly hasn’t changed the outcome.
When tracking my memory w/ task manager, having the program open on the frame (not labelling nor exporting) is using 56% of my computers memory at about 5-6,000 MB.
Just opening the file in the labels program takes 98% of my memory at over 12,000 MB, and then its the same story whilst exporting.

Let me know if this is any help!

Hi William,
it seems that you are indeed close to the memory limit. I would try to update Labels to 3.11 if possible (see here).
What I meant as a workaround in the previous post was to select and load (double click in the “Database” view) another dataset before switching to the Export tab, not to export another dataset.
If you cannot upgrade or upgrading does not help, we can have a short call and I will show you the workaround.
Best,
Federico

Hi Federico,

I’ve just installed the version linked and I’m happy to say it resolved the issue!
Thank you for all your help!

Best,
William

1 Like