cancel
Showing results for 
Search instead for 
Did you mean: 

Joining two flat files

Former Member
0 Kudos

Hi Gurus,

I'm a newbie to SAP BODS. We have a scenario, where we are joining two flat files to get single column. Both Primary and Reference file records were suppose to be in millions. We don't have any staging DB to push down to sql operations.

In this case, is it better to go for JOIN or LOOK UP?  Is there is any other way to improve performance other than staging DB?

Thanks

Accepted Solutions (1)

Accepted Solutions (1)

former_member187605
Active Contributor
0 Kudos

No, the only viable option is a staging database.

Why do you say you son't have any? Where are your DS repositories created? Use the same database environment, seperate schema or database, depending on the type.

Former Member
0 Kudos

We don't have much space to accommodate. Moreover, business is not willing to shed more on that. Any other way, to improve performance?

How about using JOIN or LOOK UP? Your views will be greatly helpful.

Thanks

former_member187605
Active Contributor
0 Kudos

I cannot recommended either option. I am afraid both will be extremely slow.

Answers (2)

Answers (2)

Former Member
0 Kudos

So, you are preferring JOIN over LOOK UP for large flat files with records in millions? Kindly let me know.

Thanks

former_member187605
Active Contributor
0 Kudos

Whom are you asking? As far as I can see, nobody's preferiing one over the otehr here. This won't work without a staging database.

Just try it out for yourself and let us know.

Former Member
0 Kudos

Thanks Dirk.We are working on that.

mageshwaran_subramanian
Active Contributor
0 Kudos

Since you do not have the staging DB , look up option is ruled out. You can join the two flat files ( as file formats).I assume you produce a output file as a result.Look for "Join ranks" in this article Joining tables in the engine - Enterprise Information Management - SCN Wiki

former_member187605
Active Contributor
0 Kudos

That's not true. I don't say it's an option in here, but in general it is (with smaller files). From the SAP Data Services Reference Guide, section 6.3.90 lookup:

"

<lookup_table>

The table or file that contains the result or value you are looking up (<result_column>). The <compare_column> is also located in this table.

"

mageshwaran_subramanian
Active Contributor
0 Kudos

Dirk , my bad.You are right.


Since you do not have the staging DB , look up option is ruled out. You can join the two flat files ( as file formats).I assume you produce a output file as a result.Look for "Join ranks" in this article Joining tables in the engine - Enterprise Information Management - SCN Wiki