Skip to Content

Can we have a comparison of Sybase IQ and Hadoop for NLS strategy?

Jan 14, 2017 at 03:54 AM


avatar image

We are trying to understand the pros and cons of both Sybase IQ and Hadoop with respect to NLS solution

10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

1 Answer

Best Answer
Volker Stöffler Jan 19, 2017 at 07:39 PM

Hi Aswini,

I have to admit that I'm more familiar with IQ than with Hadoop (and I don't even know if there's an NLS implementation for Hadoop). Maybe this helps you to rate my comments.

IQ stores structured data, Hadoop stores semi- structured data. (I consider this the most relevant difference)
IQ is a genuine SQL based RDBMS, Hadoop has SQL layers on top of a (distributed) file system
IQ uses snapshot versioning for maximum consistency. I don't know the details of Hadoop based locking / consistency
IQ stores data column wise. Semantic partitioning is possible to guide row sets to specific storage locations. Hadoop stores data row wise.
IQ is highly specialized for BI / analytic queries. Hadoop uses a generic massive parallel SIMD approach to process data. This (and the previous) aspect leads to the assumption that IQ is more efficient as long as it's treated with queries it's designed for.
IQ databases need to be backed up, Hadoop is considered its own backup through redundancy (which increases the storage footprint)
IQ utilizes multiple compression mechanisms including dictionary compression. Afaik, Hadoop doesn't know the concept of dictionary compression
IQ can implement an internal "data temperature" management through semantic partitioning. I'm not aware of anything like semantic partitioning in Hadoop and imo it would be a contradiction to the idea of Hadoop if it existed.
IQ address space is very large. Hadoop address space is considered unlimited
There is or at least used to be an SAP concept of "data temperature" where HANA is used for the hottest (1st layer), IQ for colder (2nd layer) and Hadoop for the coldest (3rd layer) data

This surely isn't complete, and, as I mentioned in the beginning, unbalanced to some degree.

HTH regardless

Volker Stöffler

Show 3 Share
10 |10000 characters needed characters left characters exceeded

Can Sybase IQ accomdate Big data scenarios ? I guess no as it is designed to store structured data .

Aswini Kumar Kalujualapati

With the Unstructured Data Access (I think, that's what UDA means) feature, it is possible to store BLOBs and CLOBs up to 2 GB inside IQ. My personal opinion is that IQ does not provide an equivalent benefit for these LOBs (compared to what it can do with structured data).


Thanks Volker for the details.