Address Profiling Unable to Export more than 25 records to Excel

[Former Member]

Hi,

I am using Information Steward 4.2 to profile about 120k global addresses.  The profiling seems to run fine and I get a result telling me I have about 8k invalid addresses.  I'd like to analyze the prevalence of each error reason but when I try to export these addresses and their corresponding error reasons into excel it only provides 25 of the 8k records with no obvious way to change it.  I have found some information about setting up a failed data repository but everything I read about this seems to be within the context of other types of profiling tasks that have rules assigned NOT address profiling.  Does anyone know how I may get all of the data in the invalid category exported?    

[Former Member · Accepted solution]

Hi Jonathan,

Would advice you to check the settings of run time profiling. I suppose you would have restricted it to 25 there.

Cheers

S

[Former Member · Accepted solution]

Thanks for the suggestion Satish.  I have adjusted the max sample data size up however the maximum there is limited to 500 or so I am told.  Additionally this isn't changing the number of profiling results I can see.  Is there some screen which shows me options for "settings of run time profiling"?  I have not seen this.

It seems this would be an obvious functionality in an application that is supposed to be designed for business users who are used to seeing their data in excel.  The more time I spend with IS 4.2 the more I am convinced that there is nobody dedicated to its UI/UX.     

I ended up doing the job in BODS and can get all of the invalid records there without issue.         

[Former Member · Accepted solution]

Hello Jonathon.

My name is Corrie Brague and I am a part of the Information Management Product Management team, supporting Information Steward and Data Quality Management.  I have a couple of additional suggestions and clarifications for you regarding address profiling and being able to analyze invalid addresses:

• You are correct in that there is a limit in terms of the results that are returned for profiling.  What is being returned is sampled data to help support you in analyzing trouble areas within your source data.  Profiling is a discovery process, supports you in getting a sense of where poor data quality may be present.  The situation is different when you are performing data assessment using validation rules.  In this case, you are able to configure a failed data repository that gives your busniess users access to the failed data via Excel.
  • Note that if you change the max sample size, you will need to re-run the Address Profiling so that additional sample data is captured.
• Specifically for address validation, there are a few other tools that you may find useful.  First, the Data Quality Advisor (DQA) with Information Steward will allow you to perform a more detailed assessment of your address data.  Take a look at this blog to get an idea of what the tool can do for you:  (see section on "Address - Invalid" and note you can export to Excel based on your current filter).  If you are interested in leveraging DQA, here are some product tutorials to get your started:. And, I think you will be delighted with the user experience of this tool as well

• Finally, if you have access to Data Services, you could run you addresses through Data Services Global Address Cleanse transform (this same transforms is leveraged to perform the Address Profiling in Information Steward) to get full out results in terms of address validation.  Even if you are not interested in cleansing the address data, you can send the cleansed results to a temporary table and access the Data Quality reports to get full details on their address data quality.  Here are some of the Data Quality reports that are available through Data Services Management Console (more information is available in the Data Services Management Console Guide).  Let me know if you want samples of any of these.
  • The Address Information Code Summary report provides record counts of each information or fault code of a specific project.
  • The Address Validation Summary report provides record validation statistics for each Global Address Cleanse transform
  • The Address Type Summary report contains record counts of each Assignment_Type field value used per Global Address Cleanse transform or Address_Type field value per USA Regulatory Address Cleanse transform of a specific job.
  • The Address Standardization Sample report shows records where fields changed during processing. The fields displayed are your input fields and the associated output fields. Status codes are on the report to indicate why the change was necessary. This information helps you to determine which fields are frequently incorrect. You can also use this report to verify that your addresses are standardized correctly.
  • The Address Quality Code Summary report provides record counts of each quality code assigned per Global Address Cleanse transform for a specific job.
• With Data Services, after running your addresses through the Data Services Global Address Cleanse transform, you can also access the Data Quality statistics tables that capture the results of the address validation/correction process to build custom reports.  These statistic tables are documented in the Data Services Reference Guide.

In addition, I wanted to ensure you that we do indeed have a dedicated development team that very much cares about the end user's experience with the product.  We strive to include usability enhancements each and every release to improve this experience based on feedback from our customers.  If you would like to log your ideas around usability improvements for the product, I would encourage your to visit and leverage the Information Steward Idea Place:  SAP Information Steward: Home.  I look forward to hearing your ideas.

Take care and please don't hesitate to reach out if you would like to have a call to discuss your use case and/or ideas.

Corrie Brague

corrie.brague@sap.com