Off Topic

Professional advice Q&A

5 replies · 1,428 views
about 7 years ago

Well most of check the forums while procrastinating at work - so I thought we could use this as a support group/technical advice space...

I'll kick things off... Anyone have any tips on preparation work before you import a shark load of data into a new database (data prep, clean up, naming conventions etc)...


VUW AFC - Victoria University Football for life

Permalink Permalink
about 7 years ago · edited about 7 years ago · History

Well most of check the forums while procrastinating at work - so I thought we could use this as a support group/technical advice space...

I'll kick things off... Anyone have any tips on preparation work before you import a shark load of data into a new database (data prep, clean up, naming conventions etc)...

Plan, plan, plan. Import small sample data set. Verify. Plan some more. Sample some more. Re-plan. Re-test. Wait until after 4 on a Friday, start data import, go to a pub you don't normally go to and turn your phone off so they can't find you.

The world will still be there Monday morning even if the database isnt:)

Permalink Permalink
about 7 years ago

That's awesome thanks very much mate! Thing is I'm in a new role and this is something I've been assigned which I'm not overly familiar with and don't want to muck it up.
So far I've exported some data and studied it. The categories are all in alphabetical order so I'm changing that and rearranging the columns in a more logical/priority-based order (salutation, first name, last name etc).
I'll then try and upload 10 contacts and see how it goes. We have a developer on standby to work with the software and make changes as we need, but in the meantime I'm responsible for the data clean up and eventually bulk upload.


VUW AFC - Victoria University Football for life

Permalink Permalink
about 7 years ago

I spent over 10 years in the ETL space doing data migrations in Oracle and SAS with datasets ranging from thousands to tens of millions. If you want to PM me some of the details I'm happy to look over it for you to see if there is anything obvious.

There is too much to say without knowing where the data is coming from, where it's going to and what you want to do with it in the middle.

Permalink Permalink
about 7 years ago

Hit start and if it all goes pear-shaped then blame Patrick!

"Ive just re-visited this and once again realised that C-Diddy is a genius - a drunk, Newcastle bred disgrace - but a genius." - Hard News, 11:39am 4th June 2009

Permalink Permalink
about 7 years ago

Thanks Martyyn will do! And yeah I'll tell my boss Patrick fudged up if the process goes pear shaped!!! ?


VUW AFC - Victoria University Football for life

Permalink Permalink