Data Conversion: The Do’s and Don’ts

Authored by Gabe Russo, Solution Architect at ImageSoft

Imagine that it is go-live Monday. Your company is scheduled to begin using OnBase, or another electronic document management system, for the first time. The level of combined excitement and nervousness is palpable. Documents and other content from the old system are scheduled to be available in the new system when team members arrive for work. Incomplete or missing content would cause serious disruption. It is in this moment you begin to ask: Did the conversion finish? Was any content missed? There are millions of documents to convert, how could I be sure? When and how will I know if anything is wrong?

I'm Fully Confident

I should have remembered – Trust but verify.

Data conversion projects are time consuming, risk-heavy, and essential. They are often conducted as an exploratory exercise wherein new challenges are born from the resolution of others. It is crucial that the outcome of processing is verified for each document. The customer should expect to know exactly what happened to his/her data during the conversion project and the responsible vendor provides it readily. Through transparency, the conversion effort is verifiable cultivating trust between customer and service provider.

Keep in mind the following points as you prepare for a data conversion. Be wary of the service provider who doesn’t.

Source of Truth

Whether using the database or the file system to dictate which content to convert, it is important to understand the differences in approach. When using the database, the service provider might miss files that aren’t represented in the database. When using the file system, the service provider may not be able to interpret important metadata contained in the database. It is better to take a hybrid approach. Use the database as the source but compare it against the file system. In favor of awareness, expose discrepancies while making a deliberate plan for content that doesn’t line up.

Determine What to Convert

Maybe the source system contains old, incomplete, or content with little value moving forward. The data conversion is great time to flush out those facts to determine what needs to be moved. Sometimes it makes the most sense to convert everything and to use retention rules to eliminate content later. At other times, it can make the conversion process more efficient to ignore unnecessary content. Regardless, be sure to define the rules for what must be converted. This will make the process of reconciliation easier to complete.

Snapshots

Ensure the conversion process is separated from the source system. Taking periodic snapshots of the source system during the life span of the conversion affords several key benefits. First, the source system is left unimpeded by the intense burden of conversion processing. Second, the conversion content is captured as a point in time representation of the source system making it easier to define exactly what content was converted and when. Third, conversion processing often requires iterative actions. Using snapshots allows iterations to occur rapidly in isolation from the source system.

Capture State

Each step of the conversion process must be ACID (atomicity, consistency, isolation and durability). Avoid implementing a solution that violates these core principles. Ensure the conversion process is designed to work on chunks of data. A conversion solution should operate as a pipeline of loosely connected events as opposed to single large unit of work.

Excluded or Errored

It must be easy to distinguish documents that were deliberately ignored from those that meet an error condition. To accomplish that means that all documents recorded in the source system must be indexed by the conversion solution. Otherwise, it will be difficult or impossible to reconcile the systems while being certain that documents aren’t missed.

Verify the import

ImageSoft includes the source system document ID as a keyword in target OnBase systems. It’s a small refinement to make reconciling the systems easy. Don’t assume every import batch is successful; Prove it.

After all, the reward of a thing well done is to have done it.

Digging Deeper into Data Conversion

Interested in understanding ImageSoft’s approach to data conversion? Want a glance at our non-traditional resume of successful conversions? Trying to anticipate what your organization should expect? Answer all these questions and more by visiting our conversions webpage.

We Want to Hear From You!

If you’re anticipating a data conversion, what is your strongest apprehension?

If you’ve undergone a data conversion, what was your experience?

Tell us in the “comments” section below or on our LinkedIn page. We read and respond – promise!

One thought on “Data Conversion: The Do’s and Don’ts

Leave a Reply

Your email address will not be published. Required fields are marked *