I forget: what’s in-memory?
By Boris Evelson
In-memory analytics are all abuzz for multiple reasons. Speed of querying, reporting and analysis is just one. Flexibility, agility, rapid prototyping is another. While there are many more reasons, not all in-memory approaches are created equal. Let’s look at the 5 options buyers have today:
1. In-memory OLAP. Classic MOLAP cube loaded entirely in memory
Vendors: IBM Cognos TM1, Actuate BIRT
Pros
- Fast reporting, querying and analysts since the entire model and data are all in memory.
- Ability to write back.
- Accessible by 3rd party MDX tools (IBM Cognos TM1 specifically)
Cons
- Requires traditional multidimensional data modeling.
- Limited to single physical memory space (theoretical limit of 3Tb, but we haven’t seen production implementations of more than 300Gb – this applies to the other in-memory solutions as well)
2. In-memory ROLAP. ROLAP metadata loaded entirely in memory.
Vendors: MicroStrategy
Pros
- Speeds up reporting, querying and analysis since metadata is all in memory.
- Not limited by physical memory
Cons
- Only metadata, not entire data model is in memory, although MicroStrategy can build complete cubes from the subset of data held entirely in memory
- Requires traditional multidimensional data modeling.
3. In memory inverted index. Index (with data) loaded into memory
Vendors: SAP BusinessObjects (BI Accelerator), Endeca
Pros
- Fast reporting, querying and analysts since the entire index is in memory
- Less modelling required than an OLAP based solution
Cons
- Limited by physical memory
- Some index modelling still required
- Reporting and analysis limited to entity relationships built in index
4. In memory associative index. An array/index with every entity/attribute correlated to every other entity/attribute
Vendors: QlikView, TIBCO Spotfire, SAS JMP, Advizor Solutions (also OEMd by Information Builders)
Pros
- Fast reporting, querying and analysis since the entire index is in memory
- Less modelling required than an OLAP based solution
- Reporting, querying, analysis can be done without any model constrains, for example any attribute can be instantly reused as fact or as a dimension. Every query with an inner join can also show results of an outer join on every column.
Cons
- Limited by physical memory
- Some scripting/modelling still required to load the data
5. In memory spreadsheet. Spreadsheet like array loaded entirely into memory
Vendors: Microsoft (PowerPivot)
Pros
- Fast reporting, querying and analysis since the entire spreadsheet is in memory
- No modelling required
- Reporting and analysis are as simple as sorting and filtering a spreadsheet
Cons
- Limited by physical memory
What did I miss and what did I get wrong – comments?