Data Hub big Raw JSON requests? Bad Idea?

[former_member473180]

I just ran 540k raw fragment items into Data Hub (in 10k blocks) and am not able to get back a Raw JSON request?

http://localhost:8080/datahub-webapp/v1/pools/HistoryPool/items/RawHistory.json

Its been over 10 minutes and I don't see a time out exception. Is this a bad idea to do this request in a browser? Should I use a JSON desktop utility?

[Former Member]

the .../items/... resource accepts already a q: Query parameter where you could check for a specific status. e.g. IGNORED to find out if your composition was not successful.

[Former Member]

Unfortunately there is no pagination for all the ../items/... requests, so the datahub returns all items of this type in a single request.

This could take a while and, depending on your JVM config, could lead to a OutOfMemory Excpetion.

[former_member473180]

is their a way to query a specific raw item for load diagnostics?

[Former Member]

If you know the raw item id (that's generated by the datahub) you could use this request:

https://download.hybris.com/api/5.7.0/data_hub_documentation/DataHubAPI57.html#_raw_items__itemId__g...

I think the only way to get the ID is to check the table RawItem in the datahub database.