Python ETL from SQL Server to Oracle and Greenplum

it_user358593 - PeerSpot reviewer

Project Description

Developed a Python ETL package that moved 5,000,000 web traffic records a day from a MS SQL Server environment out to 16 Oracle and PostgreSQL servers.  It was a critical component in Healthgrades' Patient Direct Connect product and allowed the company to merge web traffic, phone, and medical data to demonstrate efficacy.  

Difficulties

Management had to be convinced
Equipment incompatibility

Products Used

Technical Skills Used

  • Python
  • Data Warehousing
  • Performance Tuning
  • Bash scripting