|
|
|
|
| Author |
|
|
Licensed User
Join Date: Feb 2003
Location: ....down Highway 61
Posts: 6,506
|
ETL developers
You guys know everything, so I figured I would start here...
I'm looking for some conceptual input on different ETL tools. The high level overview is that I need a migration tool that can extract/transform/load xml data via multiple exposed web services. I've built much simpler VSTO plugins that hit single web service APIs to extract and manipulate data. I'm pretty sure I could push it back through a web service using something in Visual Studio, but that doesn't seem like the best tool. I'm not an ETL developer, but if I can find a tool where the learning curve isn't too high I think I can fumble through it and get it to work. Conceptually, will Clover do this, or do I need something like Informatica, Oracle, etc.? What other tools can I look into? Send me PM if you have input and would rather chat about it offline. TIA |
||
02-10-2017, 09:48 AM
|
|
|
Driver, not Mechanic
Join Date: May 2013
Location: SF Bay Area
Posts: 3,008
|
Informatica. Try the cloud version first.
|
||
|
02-10-2017, 01:08 PM
|
|
|
Super Moderator
|
ETL tools are usually focused on higher volume data transactions, is that what you need?
If you're looking for something a little simpler with low volume or non production work EAI tools are probably better. Nifi is gaining attention, by Apache, and I think will be a decent skill set to keep in your toolbag. ..but I'm in sales, wtf do I know? 
__________________
Chris ---------------------------------------------- 1996 993 RS Replica 2023 KTM 890 Adventure R 1971 Norton 750 Commando Alcon Brake Kits |
||
|
02-10-2017, 07:01 PM
|
|
|
Slackerous Maximus
Join Date: Apr 2005
Location: Columbus, OH
Posts: 18,197
|
Sorry. I'm a one trick pony. I do rails. Building APIs is easy. Ruby is very basic, I dig it. Best thing where raw speed is needed? Well...no.
__________________
2022 Royal Enfield Interceptor. 2012 Harley Davidson Road King 2014 Triumph Bonneville T100. 2014 Cayman S, PDK. Mercedes E350 family truckster. |
||
|
02-10-2017, 07:52 PM
|
|
|
Registered
Join Date: Jun 2001
Location: Sweden
Posts: 5,911
|
There are multiple products that can do this. Currently, I have been forced to use lots of IBM Datastage (mucho $$$) but have also had very good experience with Ipswitch MoveIt (less costly).
This is all basic ETL meat and potatoes that can be cobbled together almost for free (if you know your stuff) or be mega-expensive (if you opt for "brand names" and let consultants do it for you). I consider MoveIt a sweet spot between expensive/cumbersome for average guy. Quick and cheap example: Let AT trigger curl, download XML to a folder and then schedule SQL to ingest XML, transform, BCP back to a file and use AT again to upload it wherever you want.
__________________
Thank you for your time, Last edited by beepbeep; 02-11-2017 at 02:56 AM.. |
||
|
02-11-2017, 02:52 AM
|
|
|
Licensed User
Join Date: Feb 2003
Location: ....down Highway 61
Posts: 6,506
|
Awesome. Thank you all again. This gives me a lot to start looking at.
|
||
|
02-12-2017, 02:02 PM
|
|
|
1970 Porsche 911 S 
