How JustAnswer Works:
  • Ask an Expert
    Experts are full of valuable knowledge and are ready to help with any question. Credentials confirmed by a Fortune 500 verification firm.
  • Get a Professional Answer
    Via email, text message, or notification as you wait on our site.
    Ask follow up questions if you need to.
  • 100% Satisfaction Guarantee
    Rate the answer you receive.
Ask Scott Your Own Question
Scott
Scott, MIT Graduate
Category: Homework
Satisfied Customers: 3040
Experience:  MIT Graduate (Math, Programming, Science, and Music)
3546829
Type Your Homework Question Here...
Scott is online now
A new question is answered every 9 seconds

Looking for Programmer experienced in Linux, Python and

Customer Question

Looking for Programmer experienced in Linux, Python and Hortonworks Hadoop Sandbox to do the following:
1. Transfer Files (If you have any questions on this let me know)
Transfer File :
a) Check if converted file is available
b) Read HDFS target location from meta data table
c) Copy the file(s) in target location in HDFS
Error Scenario:
a) File unavailable in designated location
b) Error in reading the file
c) Error in conversion
d) Error in file writing
2. Validate File (If you have any questions on this let me know)
Validate File:
a) Check meta data table to retrieve validations for each column
b) Read file from HDFS
c) Perform validations
d) Store errors in separate file retrieved from meta data table
e) Store results in memory
Error Scenario:
a) File unavailable in designated location
b) Error in reading the file
c) Error in connecting to database
d) Error in performing validations
e) Error in writing error file
6. Transform Reference Data (If you have any questions on this let me know)
Replace File:
a) Retrieve columns and file names for code replacement from meta data table
b) Read each file for respective column
c) Lookup value with given code
d) Update in-memory dataset with values corresponding to the code
e) Flush the in memory object to database
Error Scenario:
a) Error in connecting meta data table
b) Files with code-values not available
c) Error in reading the files
d) Error in conversion
e) Error in connecting to NoSQL database
f) Error in writing data in database tables
Submitted: 8 months ago.
Category: Homework
Expert:  George Sibiya replied 8 months ago.

Hi,

Welcome to JustAnswer and thank you for utilizing our services.

I do have have experience in the mentioned technologies.

Depending on how urgent this is, I can start working on it from the 29th.

Since you have a code to work from I can estimate it to take me a day or 2.

Regards,

George

Expert:  George Sibiya replied 8 months ago.

Please advice if the above would be fine.

Customer: replied 8 months ago.
Hi George, I can send you the code so you can look at it then let me know what you think. You will need MySql database, hortonworks hadoop sandbox
Customer: replied 8 months ago.
Let me know how I should send the code to you
Expert:  George Sibiya replied 8 months ago.

Hi,

Please try http://wikisend.com/ they have 100MB upload limit.

Another one is dropox.com

If those two fail I can offer a premium service for us to exchange Google drive accounts securely.

Regards,

G

Customer: replied 8 months ago.
These are the files:Script: http://wikisend.com/download/529728/metadata8.pyData: http://wikisend.com/download/541852/we.Finance-DataFeed_example.zipInstructions:
Windows: create folder C:\files
Create landing folder C:\files\landing (the zip data file goes here)The script file goes in C:/files
Expert:  George Sibiya replied 8 months ago.

Thank you so much

Related Homework Questions