Ignore flat file columns or dump their data during load
Hello experts, we are working on the creation of BI extractors (BI 7.0), and for a couple of extractors the flat file to be loaded has several columns (that correspond to both characteristics and ratios) which are not of our interest. I have searched the forums and found several posts in which the posters were suggested to delete the columns from the flat file. On our case this cannot be done. The file comes from a preconfigured report which is not editable, so we cannot receive it without all the columns, and manually deleting the columns is not an option since this file will be automatically loaded daily through a process chain. While we can allow all the information to be loaded, we would need to delete the information of all columns which are of no use to us. Also it is worth noting that the flat file does not allow column organization that could allow us to place the useless columns at the end of the file so that when the data source reads all useful columns it ignores the rest of that line.
I thought about two possible solutions:
- Create the infocube and all the infoobjects, and in the process chain add a task at the end in which the contents of the useless infoobjects are deleted. I searched the process chain tasks but could not find one that would delete the information loaded on infoobjects. Is there a task that can do this?
- Create the infocube and all infoobjects. On the data source add all the fields, but on the transformation only map those that are useful to us. If I use this method, would the data loaded in the useless characteristics and ratios be dumped (deleted) or would it remain on them? Is there a way to delete it?
Finally, since we are talking about a good number of columns for both files, could I create only one infoobject of type characteristic and one of type ratio and map it multiple times on the data source (since we do not care about the information on those fields, whatever happens to the data written on them does not matter to us, as long as we can delete it somehow, and we will not be mapping those infoobjects on the transformation)?
Mathew Muthalaly replied
You could go with your second option. The data would first be loaded to PSA. Create transformations from PSA to Cube only for the required fields. Only the required field values would go to the cube. The PSA would have all the values.