SAP-IT internal migration from XI3 to BI4
Tags:
This article is an example on how a migration from SAP BusinessObjects XI 3.x to SAP BusinessObjects BI 4.x has been handled.
It sets up within SAP IT, as such, a SAP Internal project, from where a complete migration procedure has been executed from August 2012 to March 2013. This article will describe the different steps done and explain the different choices made, as well as highlight some hardships encountered during the migration.
My name is Adelin Sivilay and was part of SAP Global IT Predictive & Insight Analytics – BO Systems Management team, which is in charge of managing one of the official SAP BusinessObjects platforms used internally in SAP.
I have taken the lead on the whole internal migration project with the technical analysis, the planning, the execution control and also the post-migration follow-up with our end-users. As such, I have a clear vision of the project to be able to share our experience on how the migration has been executed.
First note:
This article will not explain what different tool exists to execute a migration.
For this kindof information, please refer to the following SCN articles, which served as a base for the SAP IT migration project, and will in this article act as references. Therefore, please make sure to read those articles to understand fully what will be explain in this one.
How to Upgrade to BI4.0
All you need to know before upgrading to BI4.0
Also, it will not cover any sizing/performance subject except for direct incidence on migration phase.
Summary
Preparation before upgrade
1. Inventory
First thing to look after when we wanted to migrate our XI3 content to BI4, was to do a clean inventory of what we wanted to migrate. Either in term of content amount –to serve in sizing matter-, or even in term of tools used in the XI3 landscape.
We also wanted to do some kind of cleaning before the migration: all content owners have been asked to clean unwanted or unused files to decrease the amount of data to be moved over the BI4 platform, restraining useless content to persist. To help them, we had audit reports (if requested) which summarize the global utilization of their solution. Other than that, the Solution Owners had to synchronize with their end-users to make sure that the content are relevant.
We have also informed the content owners, that the personal files, e.g Personal folders or Inbox, won't be migrated, only reports under the general Public Folders will.
We have also distributed and made available internally a document summarizung all the migration information and a small guide in regards of migration path so that each Solution Owners can refer to and also understand a bit better the context and steps of this project.
In term of tools, we had the complete set of BusinessObjects suite, as followed:
- Web Intelligence
- Crystal Report 2008
- Explorer
- Xcelsius
- LiveOffice
- BI Mobile
In term of connectivity, which is also important to list because of BI4 tool compatibility change, we had the following types:
- ODBC (IBM DB2, MS SQL, …);
- Flat DSN (Attunity);
- SAP BW.
XI3 Tool/Technology | Backend Type/Connectivity |
Crystal Reports 2008 | SAP BW with MDX |
SAP BW with BW Query | |
ODBC (MSSQL, DB2) | |
XLS | |
Optim Attunity | |
Explorer | UNV on SAP BW with BW Query |
UNV with ODBC | |
XLS | |
Web Intelligence | UNV on SAP BW with BW Query |
UNV on ODBC | |
Xcelsius | SAP BW with BICS |
Live Office on WebI | |
Live Office on Crystal Reports | |
Mobile BI | Multiple |
As for the content, we have moved over 22 solutions pided in 9 LOBs, for an approximatedamount of 800 objects (documents, universes, connections).
As a side note and information, we had on our XI3 productive system over 3500 distinct users connecting a month for 80000 registered.
2. Tool Migration Path analysis
As specified in the “How to Upgrade to BI4.0” article, some changes have been made in BI4.0. As such, if you plan later (after moving all your content as a 1-1 migration as suggested) to switch to those new features in replacement of the old ones, you will need have an eye on which tool you’re going to use in the future and what conversion path makes sense for your end-users.
For instance, if you used Crystal Report 2008 on top of a SAP BW backend (hence MDX Connection), you will either have the choice to stick with Crystal Report 2011 (keeping MDX) or to leverage the new Crystal Report Enterprise that use the new BICS Connectivity. Another example would be the simple question of converting to UNX format universe or keeping UNV format along? Or use both? But the latter would be a pain to maintain.
In our project, we have decided to prefer the new technology when it was feasible (e.g. UNX over UNV) and highly recommended the usage of new tools.
We have decided to not support UNV as most as possible, or at least, not having a mix of both format within a same solution to be able to easily maintain/support any issue coming up in the future.
Below is a sum up of what has been instructed/recommended to our platform Power-user and developer.
XI3 Tool/Technology | Backend type/ Connectivity | BI4 tool recommendation |
Crystal Reports | SAP BW | Crystal Reports Enterprise with BICS connectivity |
ODBC (MSSQL, DB2) | Crystal Reports Enterprise with ODBC connection or UNX universe | |
XLS | Crystal Reports Enterprise | |
Optim Attunity | Crystal Reports 2011 | |
Explorer | SAP BW | No direct support as of BI4.0 |
ODBC | Explorer with UNX universe | |
XLS | Explorer with XLS | |
Web Intelligence | SAP BW | Web Intelligence with BICS Connectivity |
ODBC | Web Intelligence with UNX universe | |
Xcelsius | SAP BW | Dashboard with BICS connectivity |
Live Office WebI | Dashboard with UNX Universe/BICS connectivity or Dashboard with Live Office WebI | |
Live Office Crystal Reports | Dashboard with UNX Universe/BICS Connectivity or Dashboard Live Office Crystal Reports | |
Mobile BI | Multiple | No changes |
3. Upgrade Management Tool
As explained in the “How to Upgrade to BI4.0”, Upgrade Management Tool is the premium tool to migrate all your content from one platform to another. This is as such the main tool used during the “technical migration” (moving content phase) paired with Import Wizard Tool (BO XI3 platform) to make BIAR files.
Upgrade Management Tool can run several scenario and mode. And we, as hinted above, decided to go with BIAR files, instead of a direct “Live-to-Live” scenario for the following aspects:
- It’s safer (no risk to break one or the other system during transfer);
- BIAR files can be used as backup files just in case (life savior, I can guarantee that);
- No downtime required on any platform during the move (this is crucial if your end-users are still using the platforms during the migration period, especially if you have a big amount of data to move out and as such would require several downtimes in the Live-to-Live scenario);
- Higher control on what’s being moved;
- Faster troubleshooting in case files are corrupted as only what’s has been selected in the BIAR would be impacted.
We have also decided to split the migration by solutions. As such, we did an incremental migration instead of a complete upgrade:
- Each solution has its own pace and LOBs, thus, the time allocated or availability for migration can differ from one solution to another;
- More granularity over the migration;
- Distinct Backup for each singular solution at the migration time;
- No impact on solutions not yet ready to migrate.
BI 4 setup
This part will cover information regarding the system itself. This can be referred as the Step 2 of the « How to upgrade to BI4.0 » article.
1. Landscape description
In SAP, we have a full landscape dedicated to BO XI3 operations. By full landscape, we mean that we cover the following setup:
Development > Test > Quality > Production
As understandable, we wanted to move this entire setup to the BO BI4 version. However, even if possible, we didn’t run a side-by-side installation. For two reasons:
- It’s not recommended;
- We already had a full landscape dedicated to BI4 operations (so we were already maintaining two landscapes for it not to be a drawback for the UMT scenario choice).
As such, all installations, security framework, sizing, performance tweaks has been done prior and adjusted for the migration.
Please be alert of your FRS disc space consumption. This is the most critical metric to monitor during the technical migration phase (content move).
2. Landscape preparation
Knowing that we have this separate installation of BI4 running, we had the choice between either migrate all content from XI3 as a parallel migration or, in a more linear way.
The purpose of a “parallel” migration would be to have a complete mirror of what is in your XI3 platform to the BI4 platform, like a save state. However, this would also mean 2 things:
- To have the same number N of systems in the two landscapes;
- To execute N-time the content migration for the N number of system composing the landscape, this can create tremendous workload.
In regards of the amount of content we had to migrate and to be in line with our desire of cleanup and control over the migration, we have opted for the Linear Migration. This allowing us the following:
- Choose the Source system (either development if it has the most up to date version of a solution or productive version; or even a mix of both: first productive version then followed by an incremental migration of some documents from development system);
- Less BIAR to be generated;
- Only one Destination system (BI4 Development system);
- Manual adjustments (the recommended tool update changes as identified in chapter 1.b - see also chapter 3.b) to be applied on only one system instead of the 4 (especially when we do not allow any direct modification on Production), so no task redundancy.
- Complete Change Management process can be reapplied on top of this migration: control anew what’s going/allowed to productive system.
It would also benefits upon Ad-hoc reports that have been created on the XI3 platform productive system -as they were integrated in the migration as long as they were located in the Public Folders-:
- Platform teams can control back the quality and performance of those reports that were created directly by the business;
- Integrate those ad-hoc reports to the "standard" folder of the solution: in our BI4 landscapes, we have introduced two folders within each solution. A first folder called "Standard" where are stored the reports developed from Dev system and considered as the "standard" of the solution hence where platform team ensure Support (read access only, no edit rights). And a second folder called "Business" where Power users are able to create, edit, delete ad-hoc reports on Productive system for their own needs.
Migration phase
This chapter will explain how we have split and handled our migration process. For a better understanding to our power and end users, we have made split the migration into 3 steps having each its own purpose and its own actors.
Here is a schema of those steps:
1. Technical Migration
The technical migration phase is the step where the contents are being actually moved from the XI3 system (development or production) to the BI4 Development system.
However, it does not only contain this major step but also intervenes at first the Solution area creation: creating the new solution’s folder aligned with the new security framework, creating the DSN/ODBC connections, the BO connections, user group mapping, user assignments, etc.
Each migrated solution has been treated as a new oncoming solution: this was the most effective way to ensure the alignment with the new security, folder architecture and also the new naming convention of our BI4 landscape. The only difference with a “real” new solution is that the content will be moved over from XI3 in our case.
When the solution area has been successfully created, we used Upgrade Management Tool to bring over the needed content using the strategy explained in the previous chapter. Was left the connection change to Development backend (if the content came from the XI3 Productive system, then we need to change the target backend source from productive to development as the content are moved to BI4 Development system).
Prior to the content move, we have made sure to have from the Solution owner the source system, the migration date approval (that we have discussed internally beforehand to align with the migration timeframe) and of course, approval of migration so that every party are aligned.
The main actor of the Technical Migration phase was the Platform management team: BOE Administrator and Support Team.
2. Manual Adjustments
The manual adjustment phase comes right after the Technical Migration and on top of the Development system. As already stated in a previous chapter, this step is the one reserved for the solution to adjust to the new BI4 recommendations (e.g. convert to UNX universe). It's strongly recommended to proceed with this step only after completing your 1-1 migration, basically, just pushing everything back to Production as it was on XI3 as soon as possible, and then modify reports to use new features from Development system. Like this, you will ensure a small and smooth migration period.
In our case, thanks to the experience already aquired from BI 4 validation programs and ramp-up as well as access to SAP experts, we took the controlled risk to do both in the same project. But again, this is not the recommended practice.
This step was a great opportunity to test the new features and tools offered by BI4. As one of the first customer to move over and use the new tools, we were more likely to see product defects during this phase. Most of them have found either workarounds or have been fixed in new patches in collaboration with our Product Group.
As mentioned earlier, we tried as much to force solution owner to convert to the new tools and stick to the recommendation, however it couldn’t always be possible mainly for the following reason:
- This step involves primarily the Solution owner and the Development team. As such, for the Platform team, this step if the most hard to quantify either in workload and time needed as it greatly depends from one solution to another.
- Too many reports to convert for small bandwidth at this time;
- Conversion not always intuitive (ex: CR2011 to CRE against SAP BW);
- No big ROI.
This step involves primarily the Solution owner and the Development team. As such, for the Platform team, this step if the most hard to quantify either in workload and time needed as it greatly depends from one solution to another.
3. Change Management Process
When the Manual adjustments phase is over, the solution owner nominates the solution to go towers production. This is part of the usual Change Management Process. As such, the solution will reach Test and Quality system, where it will be validated and approved prior reaching Production.
When a migrating solution reached production, we have cut all access of this solution on the old XI3 landscape to make sure of the adoption of the new BI4 system by the end-users and thus closing its migration.
Involved actors: System owner, Platform management team and Support team.
Additional highlights
1. Technical perspective
Technology | Description | Fix/Workaround done | Target Fix/Explanation |
Webi | |||
UNV OLAP BW MDX | Performance difference between XI3/BI4 (slower in BI4) | SAP Note 1763544 | |
UNV ODBC | Converting to UNX: missing filters and unresolvable object error | ADAPT01645276 | |
UNV ODBC | Context not working properly (always being prompt after transport) | SAP Note 1720718 | BI4.0 SP2.19 or above |
Explorer | |||
UNV ODBC Datasource | No support of UNV | Conversion to UNX | Product limitation. Working in BI4.1 |
UNV BW Datasource | No support of UNV. Cannot be converted to UNX | Add an intermediary ODBC datasource between BW and BOE to be able to create UNX universe | Product limitation. Working in BI4.1 |
Any | Indexing hanging in running state causing Indexing Server to restart/hanging | Increase the Memory allocated to Explorer.Indexing service | Ressource consumption was high (1 infospace for example takes 1 hour to complete itself) |
Crystal Report | |||
BW Datasource: BW driver | Converting to MDX driver: there is no automatic 1-to-1 mapping of fields | Manual mapping of each Crystal Report fields to switch to MDX Driver instead of BW Driver in CR2011. | BW Driver is no longer supported in CR2011. Depreciated technology. |
All | Unable to get the Prompt values (loading hanging then timeout) | Need to check the printer option within the report. "No printer" would be best as the printer configuration might be different | |
All | No option to open hyperlink in a new tab | BI4.0 SP6 or above | |
Dashboard | |||
LiveOffice Webi | Timeout when trying to add a Webi Object or refreshing a webi object when number of users registered is exceeding 10 000 | BI4.0 SP5.3 or above BI4.1 or above | |
LiveOffice Webi | Intermittent "Failed to get Document Information (LO 30270)" error when refreshing several webi objects in one dashboard | Increase Webi Max connection in each Webi processing Server and close all connections after transaction in Universe's connection settings | LO is creating a lot of parallel requests at the same time per WebI object which can quickly cap the default limit |
Live Office | Migrated XLF is taking time to publish | SAP Note 1747458 | BI4.0 SP5 or above |
LCM | |||
Error 500 or infinite loading when import/export of LCMBiar | Split the LCM job in several smaller jobs / Insert less object in the job | LCM Jobs is too big | |
Lot of temporary files left in filesystem | SAP Note 1508918 | BI4.1 or above | |
Cannot delete folder inserted in a job | Recreate job or delete folder after transport | Product issue in BI4.0 SP4.2 only. Fixed in BI4.0 SP5 | |
Platform | |||
FRS | File Repository System storage space full | Increase the size of the filesystem storage | |
Connection server | DSL Bridge (BICS connection) High Memory Consumption | Increase dedicated Memory for the server | |
Tomcat | SDK "failed to get document output file" when opening Webi Report | Set XX:ThreadStackSize=1024 in the webapp | |
Security | Webi UNX Performance slow for regular Users | Missing right to "download locally connection" |
2. Management constraints
One of the pain points is to be able to leverage enough traction in every involved team for the migration. Time can also be a decisive no-go for business to get involved in a migration: QEC period for instance.
As such, it’s critical to define milestone for every solution and agreement on the migration timeline. We had tremendous reluctance from our end users to switch to the new BI4 system due to those tight time periods that exist between each QEC and also because we didn’t define firmly a target date of move that can’t be delayed.
It’s also important to remind the purpose of this migration, whether it is to leverage new tools, possibility and knowledge or simply to reduce internal cost so that everybody feels concerned regarding the migration. Communication is key.
Finally, another point to not omit is the learning: BI4 brings new front end technology, new platform services, etc. As such, it’s important for the platform team, the report developer team and the end user to get the adequate trainings to be able to profit fully of the BI4 functionalities.
Benefits & changes
The move to BI4 has bring a lot of new features as already specified in previous chapters.
It's now easier to develop on top of SAP BW, for example WebI, thanks to BICS connectivity (no more OLAP UNVs are needed), same thing in regards of Dashboards on BW. However, feature aside, in term of performance, our end-users have noticed better performance and stability. We had good feedback especially with Crystal Reports and WebI paired with UNX.
Administration side, thanks to this move, we have been able to implement the Online Backup feature, which permit us to no longer have planned downtimes on weekends for backup purpose, greatly appreciable for high availability and, of course, to enable the whole world's timezones.
The new Monitoring feature has made us more reliable thanks to the different watches we have set and the Audit enables us to provide better statistics for the Solutions' Usage.
Please also note that somes process need to be changed, especially for Change Management as Import Wizard Tool is no longer available on BI4 but "Promotion Management" or also called "Life Cycle Management (LCM)". As such, you need to rework your transport procedure around this new tool and also , of course, get familiar with it.
Conclusion
Our internal migration took 9 months to finalize. However moving the content (technical migration) was not the longest part and has been done in a two-month timeframe. What took us time were majorly the Change Management phase (step 3) and the acceptance for some solutions to move over to the BI4 landscape due to, as said in the Management constraints chapter, QEC periods and also product bug resolution.
As such, it’s really important to prepare beforehand your migration, not only regarding the technical setup but management wise too. Once your plan is concrete and ready, migrating shouldn’t be that complicated as long as you stay vigilant regarding issues that might occur.