MaxDB use for datawarehouse storage and B.I. applications

[Former Member]

Hi,

i'm thinking to use MaxDB for datawarehouse storage and B.I. applications, to use with the open source BI Pentaho, anyone know or wish to share experience about use of MaxDB for datawarehouse star schemas? and how is the performance with large sets of data, any strategy to make "partitions of data" like use table by year or schema by year? any parameter to set on db server to work better with this type of application?

any tip are welcome.

best regards

Clóvis

[lbreddemann · Accepted solution]

Hi Clovis,

SAP BI/BW is used on MaxDB and not too bad on it.

With MaxDB => 7.6.04 and =>7.7.06 several special features ([WIKI BW-Feature Pack|https://wiki.sdn.sap.com/wiki/display/MaxDB/MaxDBBWFeature+Package]) have been introduced that greately improve the performance for BI Queries.

However, these features are specifcally designed to be used by the SAP BW component and not well documented for the "outside world" usage.

The features include physical and logical clustering of data in the tables, packed storage of 0 and NULL values as well as special optimization knowledge for the optimizer so that it can handle star queries more efficiently.

As I know that you're pretty much into developing with MaxDB I don't want to stop your interest too early here - but I believe that it's going to be rather frustrating to try to buildup your own MaxDB BW if you don't know how the features are meant to work.

best regards,

Lars

[Former Member · Accepted solution]

Hi Lars,

would you explain better what you mean with "to try to buildup your own MaxDB BW" ? and exists a chance for "dark side" know this documentation? or better is i really forget MaxDB (put in trash years of use) and go to Postgres or Ingres database. Please be honest, dont give me a think of someday my wishes will be attended. i really wish the positive sense ;).

best regards

Clóvis.

[lbreddemann · Accepted solution]

> would you explain better what you mean with "to try to buildup your own MaxDB BW" ?

Sure. I mean "If you want to build a datawarehouse with MaxDB then it's going to be difficult since you're not inside of SAPs BW developer group".

>and exists a chance for "dark side" know this documentation?

Hmm... I dunno... I'll ask if I am allowed to show some examples on how to do things here.

>or better is i really forget MaxDB (put in trash years of use) and go to Postgres or Ingres database. Please be honest, dont give me a think of someday my wishes will be attended. i really wish the positive sense ;).

Well, SAP does not move away from MaxDB.

We don't move to PostgreSQL or Ingres - all our products (including BW/BI) run on MaxDB.

In fact MaxDB was heavily extended to support all that.

So, yes, honestly it makes sense to stick with MaxDB.

[Former Member · Accepted solution]

Ok, then

what product of SAP i must purchase as a developer that can give me support and access of specific notes and documentation for MaxDB? i talk with people of SAP here on brazil but none of then know a way to have this support. (maybe we are too small?)

best regards.

Clóvis

[lbreddemann · Accepted solution]

> what product of SAP i must purchase as a developer that can give me support and access of specific notes and documentation for MaxDB? i talk with people of SAP here on brazil but none of then know a way to have this support. (maybe we are too small?)

Hmm... I'd guess you would need to buy a license of SAP BI to get the stuff - but then you don't need it anymore...

Tell you what: I got the permission to demonstrate some specific DW features, so this is what I'm going to do.

I'll think I make up some Blog or Wiki pages or something like that to illustrate the features.

This will take some time (of course).

So the best would be you start by giving me something to work with.

Some stripped down example of what you want to do.

Do you think you can do that?

cheers,

Lars

[Former Member · Accepted solution]

Hi, Lars,

Very good news, very, very thanks for your iniciative.

yes i can do that, as i will use Pentaho, pentaho have some samples and tables of star schemas, i will get that tables and data, to check what will be usefull for us to start this, then we can test things.

best regards.

Clóvis

[Former Member · Accepted solution]

Hi Lars,

the material you want i put in follow link:

http://sites.google.com/a/supridatta.com.br/maxdb/starschema

please let me know if you need more information.

best regards.

Clóvis

[lbreddemann · Accepted solution]

Hi Clovis,

thanks for the material.

It's going to take some time for me to make up some demonstrations with that, so please be a little patient.

If it's OK to you I'm going to make some blog entries on that as the forum is currently very limited concerning the formatting of longer posts.

regards,

Lars

[Former Member · Accepted solution]

Hi Lars,

i think that is ok to create a blog for this.

just a doubt, i can install MaxDB and start using, and after i can migrate or update with features for datawarehouse? or i must install having that features enable?

regards

Clóvis

[lbreddemann · Accepted solution]

> just a doubt, i can install MaxDB and start using, and after i can migrate or update with features for datawarehouse? or i must install having that features enable?

No, you do always have all features installed and usable with MaxDB.

We don't sell db features. We're not a DB company

As an overview you might be interested in the BW Featurepack presentation that is available on the MaxDB website: http://maxdb.sap.com/training/ -> [BW Feature Pack|http://maxdb.sap.com/training/Miscellaneous/SWS2007-BW-FP.pdf]

regards,

Lars

[Former Member · Accepted solution]

Fine, i will read and wait your blog,

best regards

Clóvis

[lbreddemann · Accepted solution]

Hi Clovis,

please do the export table by table, since currently the loader needs more information to import the data, than what's in the export folder.

At the moment I cannot import your tables.

Thanks and best regards,

Lars

Hi again - no need to rexport the tables.

I had a look into your data model and I will make up a similar one (with english names) by myself.

regards,

Lars

-

---

Marketing: if you're not already booked for October 27-29 this year and you happen to stay in vienna and you keep asking yourself what to do ... then get your ticket for [SAP TechED 2009|http://www.sapteched.com/emea/] and make sure to attend my [MaxDB session|http://www.sapteched.com/emea/edu_sessions/session.htm?id=237]

[Former Member · Accepted solution]

Hi, Lars,

sorry my delay, i just read your message now, but there is another way too,

if you download http://sourceforge.net/projects/mondrian/files/mondrian/mondrian-3.1.1.12687/mondrian-3.1.1.12687.zi... inside this ZIP have a file demo/FoodMartCreateData.sql and a demo/access/MondrianFoodMart.mdb that have same star schema inside.

maybe this helps more you in creating an English version.

and if you wish have a link http://mondrian.pentaho.org/documentation/installation.php#2_Set_up_test_data that demonstrates how to run scripts that create the sample data on many databases.

if you like the idea, i can test this way here.

best regards.

Clóvis.

[lbreddemann · Accepted solution]

Hi Clovis,

sorry for the long pause in this thread - unfortunately I've found no time yet to prepare any demo of the BI features.

Due to the summer vacation time there is much less time to be saved for such things at the moment.

But - hang on - I don't forget about this and will come up with something.

best regards,

Lars

[Former Member · Accepted solution]

Hi Lars,

I'm here and hungry about that information, i started the use of Pentaho with MaxDB, now experience some bottlenecks with a fact table that have 2,971,220 rows and 1.1G in size including dimensions, the data cache area is configured to use 2G, that give me a data cache hit rate of 100%, but i think that i need more info to increase the speed.

Well, dont worry, i will wait, and one more time, thanks for your help and attention.

best regards.

Clóvis

[Former Member · Accepted solution]

Hi Lars,

any news about this topic?

best regards

Clóvis

[Former Member · Accepted solution]

Hi Lars,

Any news about this thread?

regards

Clovis

[Former Member · Accepted solution]

Hi Lars,

long time, uh,

I can have any hope about you writing this blog? or SAP changed your mind and I must forget this?

best regards.

Clóvis