Oracle Retail MOM: Data Migration - Key Strategies

Hello Oracle Retail Fans. I’m restricting this article, since it was originally written. Oracle released Data Conversion Toolkit since RMS 12.0.3 for the automatic data conversion from the legacy system. The latest version of this tool is available in RMS 13.x version too. Though we have this Toolkit available, still we need to write multiple other scripts for the data migration as toolkit doesn’t support migrating dynamic/transactional data. 

Please do note that this post is a continuation of Oracle Retail Merchandising System Deep Dive article and if you are a beginner, you may want to visit Oracle Retail Merchandising System Overview article before proceeding on this article, however, it is not completely necessary.

Data Migration should be treated as a small parallel project along with the complete implementation and can be broadly categorized in 4 phases:

1.   Requirement Definition and Design Phase  

This is the most important and crucial phage. In this phase, the detailed Data Mappings (with legacy and current system) and the strategy will be required in order to begin the detailed design process and also to produce conversion detailed designs and document procedures (manual and automated data conversion) to source, clean, convert and validate the data to be converted from the legacy systems to RMS.

Identifying the actual data requirements for the target applications along with all associated data mappings, assumptions, issues, process flows, data volumes, dependencies and data cleansing requirements are the main task in the Requirement Definition and Design phase. Following is the brief approach, should be followed for this phase:

1. Determine Data Conversion Scope based on the designed functionality of RMS
2. Determine the All Configuration Options
3. Determine Conversion Environment Configuration Strategy
4. Specify Target Data Requirement and the mode of conversion for each table’s field in the functional area.
5. Identify the data source (existing source systems) that will be used to populate the Target applications.
6. If data is not available in source systems, it should be created manually in flat files or in spreadsheets.
7. Identify and compare the Data Volume and Complexity of the data structures (source and target systems).
8. Using the target data requirements which are already identified, map the fields to the current source system.
9. Determine if any transformation required between source and target data systems.
10. Develop a data conversion functional design document for each functional area. Document should specify overall conversion process (automated/manual) for each functional area.
11. Document specifies all the data extract and transform requirements based on the mapping exercise and also provides the solution approach to all conversion functional issues.
12. Identify all the pre and post dependency of each functional area conversion.
13. Document all default values and any data validation requirements.
14. Understand data model of each functional area in source system.
15. Document the data cleansing requirements available and identified at this stage.
16. Create validation approach for conversion results done my Merchandise toolkit and manual conversion.
17. Plan all the validations which are not supported by toolkit with proper business acceptance.
18. Plan all the static/transaction tables which need to be reloaded each time in every environment.
19. Get the Sign Off on each functional design document.

Key Deliverable for this phase:

• Data Conversion Functional Design Document
• Data Model Table Matrix

2.   Build & Test Phase  

This stage is to produce a set of custom modules (for all automated conversion processes) based on the design documentation from the design stage andtest the data conversion processes and custom modules cutover. Key objective of this phase is to produce data conversion scripts based on the data conversion functional designs, review and also give an initial view of conversion and data validation process execution times which will be further validated during subsequent passes of conversion. The recommended approach for this phase is:

1. Review Data Conversion Functional Design
2. Develop Trial Scripts which will simulate the actual cut over
3. Prepare the target (Oracle Retail) environment. Install the Merchandising Conversion Toolkit and configure it.
4. The source (current systems) and target (Oracle Retail) environment preparation will be coordinated through the Infrastructure Team.
5. Preparation includes activities such as making sure the space required is available and the current systems’ data can be extracted, and ensuring the Oracle tables are in place
6. Transformations into the flat file format specified by the Toolkit if required.
7. Build and unit test conversion modules according to the specifications of the data conversion functional and technical designs
8. Execute Data Conversion Proces.
9. Adhere to the logical build sequence e.g. tables followed by packages
10. A complete batch cycle should be run at this stage which helps to validate data integrity
11. The Build Pack should unit test conditions and a code review
12. Validate the Data Converted / Results, Identify and prioritize enhancements/fixes to be made to the conversion
13. Repeat the Trial using the original scripts after applying the enhancements/fixes.
14. Following successful unit testing, the conversion Build Pack can be signed off.

The entire source data for data conversion should be broadly classified into the three types described below:

Reference – the application’s configuration data, which controls how the application functions and is referenced during general operations (e.g. calendar, functions, error messages, and system options)

Foundation – the business data which is generally non-historic and has a low rate of change (e.g. hierarchies and suppliers) also known as Master Data.  This may be further classified as dynamic or static, depending on the rate of change.  Static data changes hardly ever or rarely.

Transactional / Dynamic Data – the business data that has a higher rate of change and contains transactional information about the daily functioning of the business (e.g. purchase orders, transfers, and inventory management transactions)

   

Key Deliverable for this phase:

• Conversion build pack
• Unit Test Results

3.   Conversion Cutover Phase  

This phase is to convert the legacy data required by the Oracle Retail Suite to the production database in phase wise. The objective of the conversion and cut over phase is to populate the target applications with the right data in a timely manner, and gain business acceptance of this data. The period of time between the initiation of the first data conversion activity (generally an extract from a legacy system) and go live is referred to as the “Cutover Window”. Below is the recommended approach for the conversion and cutover:

1. Prepare the source (current systems) and target (Oracle Retail) environment.
2. Execute end-to-end Data Conversion Process. Include pre-conversion cleansing activities, extracting current systems’ data, executing the conversion modules and optional batch processes, executing the Oracle Retail Merchandising conversion tools, and post-conversion cleansing activities
3. Database backups must be taken in some critical points of the conversion process.
4. Each functional area must be tested to ensure that no programming errors in the conversion process.
5. The attributes of each converted business objects are preserved after conversion. For instances, use the standard inquiry form to review the attributes of a business objects or run the standard report to list the attributes of a specific type of business objects. E.g. Suppliers
6. All access to the legacy system must be prohibited during the cutover period so that the data between legacy systems and new application system is synchronized. With the exception of the conversion modules themselves, all access to the production environments should be read only during the cut over window.
7. The cutover period must be kept as short as possible to minimize the impact on the daily operation. . However, the actual duration is determined by the overall execution time of the whole conversion process.
8. DBA and Network Administrator must be available during the cutover period.
9. Create transactions on the converted business objects to make sure that they can be processed by the new application system. The transaction result must match the expected result.
10. Obtain Business Acceptance.
11. Develop the contingency plans on different issues that may happen during the trial conversion and cutover conversion. The action item must have the following information.
• Potential Issue
• Action Plan
• Responsible Party/Person

4.   Business Acceptance & Go-Live  

Once the business users have certified the conversion result meeting the pre-defined acceptance criteria, the new system can go into the production. Business acceptance phase will involve the following phases or major activities:

1. Data acceptance preparation – Define and develop all scripts, reports and processes required to execute the data acceptance phase.  These will be defined and developed as part of the conversion build phase
2. Data validation – Data tests, validation techniques and checking to be done in order to validate the data converted. The data validation will be done using the following techniques / tests:
• On-line application testing – users will test the on-line application to ensure that it performs as required with the converted data.
• Validation built in to the Merchandising Conversion Toolkit. 
• Data calculations – various aggregations of quantitative data can be performed on both Oracle Retail and source applications and then compared
• On-line samples – on-line sampling in RMS and the source application(s) will be performed to check for errors / inconsistencies
3. The degree of testing will be based on the sensitivity of the data and the volumes to be tested
4. The Business representatives, who will in turn sign off the data, will perform most of the data validation activities supported by the data conversion team
5. Business acceptance / sign off should involve compiling all the output from the data validation activities.  These will be reviewed and signed off by the Business executives indicating that the data is accurate and will be accepted for ongoing business operations
6. The detailed Business acceptance requirements will generally be defined during the build phase after the detailed conversion requirements have been completed and agreed.  The Business acceptance requirements will normally include the following:
• The data to be validated, by functional area
• The tests / validation techniques that will be applied to each data entity
• The resources required for each test 

Pls feel free to comment if you have any questions. Let me know if any additional detail required.

— Share It —

• Facebook
• Twitter
• Google+
• Linkedin
• Pin It

About Nagesh Mishra

Nagesh Mishra - A Passionate Oracle Retail Certified Professional with more than 17 years of overall experience in IT industry and more than 15 years of domain expertise in Oracle Retail Applications. Worked extensively in diversified fields of Product Implementation, Business Consulting, Pre-Sales, Application Software Development, Maintenance and Support and Re-Engineering Oracle Retail projects.