Failed Data Reporting - Alias Names across Environments

[Former Member]

We have Info Steward on 3 environments : Dev, Integ, Prod.

There are also have 3 Failed Data Repositories, 1 for each of the above.

Now we want to do some custom reporting against the FDR.

Normal process would dictate to develop these views on the Dev FDR, then migrate to Integ and through to Prod.

However, what I am seeing  is this.

I have table ECC.MARA in my IS project, and on Dev, the failed data gets dumped into FDR with alias "MARA_123"

But if I now look in the same project on Iteg, the alias for the same data tables is 'MARA_345'.

Again, looking in prod, the alias if different again 'MARA_567'.

Obviously this make it impossible to create a view against the alias in Dev, and simply migrate it through to the other environments.

Yes, of course I can work around this, but the effort, in my opinion, is way way above what should otherwise be a simple process.

(I'm currently looking at scripts that accept FQ table attributes and then look up alias to dynamically build views/procedures from template)

So now I am curious ... do other people see this same phenomenon ?

Is there a mechanism to 'fix' aliases across environments ?

How do you work with it/around it/fix it ?

Appreciate any enlightenment on strategies for custom reporting in this context !

Many Thanks

Simon.

[adrian_storen]

Simon,

I have seen the same behaviour.  By design, the failed rule tables appear to be generated with a unique Id to avoid potential conflicts with existing objects.  This is painful when doing what you suggest although I have not yet formally received a requirement for custom DQ rule failure reporting that would necessitate a Universe on the failed rules repository/ repositories.

I have not tried building my own tables of the same name in each environment and this seems risky and would shy away from this approach.  So you may need to look at a DB view solution where a DB stored proc/ package auto generates the view based on names in the FD and FR tables and then auto grants this to the Universe connection user.

I previously raised idea D19898 as the failed rules repo does not update when you add / change an existing column, however I do not think I raised another for this issue.

Be good to know what solution you end up with.

regards

Adrian

[Former Member]

Ah yes . 'why' would one want to do this in the first place.

One reason is to identify 'repeat offenders'.

Lets' say material 12345 failed a rule today, in October.

The usual action is that a team will go away and 'fix' the bad data.

However, what if that rule also appeared in June's audit ?

That would suggest that the fix being conducted is not the actual root cause of an issue.

Knowing that the same data errors are repeating, tells us that we have to look deeper into what is actually causing the issue in the first place ... rather than constantly patch the same symptoms,

Many other good uses, such a 'life of defect'  - giving an indication of how long it takes teams to clean up anomalies.

[adrian_storen]

I agree with you Simon.  I know the requirement for custom DQ reporting is coming for the same sorts of reasons.  I raised an idea (D17355) about the DQ lifecycle (ie how long does it take to resolve from when first identified) although I'm not sure whether (or when... or if) this is planned for DI.

Your idea about repeat failures seems similar but different (frequency as well as duration).  It would highlight the cases when integration / MDM is more likely the cause or solution.

You could always build a DI rule for duplicates (is_unique) and bind to the relevant field in the failed rule repository.  This would just be a different binding in each environment.  I think the DB views and custom Universe might be a better way though.

[Former Member]

You could always build a DI rule for duplicates (is_unique) and bind to the relevant field in the failed rule repository.  This would just be a different binding in each environment.  

This is my problem Adrian !

If I import the FD table (123_FD)  into my project in DEV, then all is fine and good.

I can bind to it and check for duplicates.

However, when I migrate that project into Integ, there will be no 123_FD in integ, because that table in integ will be called 456_FD.

And as Integ is locked down (as a good integ environment is)  I have no option to change the binding.

[adrian_storen]

I understand - however re-binding the rule in each environment would be less difficult than building database components and a Universe.  I was just trying to offer an alternative solution to the known issue of different object names in the database.

[Former Member]

Ah, ok .. sorry Adrian I misunderstood your intent

In that case - thanks for the suggestion!

So I guess that brings me to the latter part of my question(s)

"How do you work with it/around it/fix it ?"

My company has quite strict policies, whereby the Integ & Production systems are heavily locked down.

For example, we can import projects, execute tasks, view scorecards .... but very little else.

It's an absolute no-no to change any part of the 'design' of a project by altering views/rules/bindings etc.

Sitting in my isolated box... I guess I am curious as to how other organisations set up their environments.

Maybe especially as far as the FDR goes.

The FDR has very minimal effect on the actual application itself.; but still get's caught up in this process.

Would be interested to hear how anyone else has their Dev-Itg-Prod environments set up for Info Steward & the FDR.... ?

Many Thanks