|
|
|
|
| Author |
|
|
Registered
|
Always the same data in the same format with the same number of rows. If not, it is supposed to get rejected and tell the sender why. So if row 33 is missing it gets sent back saying row 33 is missing. Very complex stuff, I know.
__________________
Brent The X15 was the only aircraft I flew where I was glad the engine quit. - Milt Thompson. "Don't get so caught up in your right to dissent that you forget your obligation to contribute." Mrs. James to her son Chappie. |
||
03-10-2021, 02:01 PM
|
|
|
Feelin' Solexy
Join Date: Oct 2003
Location: WA
Posts: 3,805
|
As a former software engineer/consultant/solution architect myself I usually take the developer's side on these things... I've definitely had to have many talks with clients and stakeholders etc to explain to them why their seemingly simple request is not simple at all to implement.
That being said, for this project I keep hearing more and more info (e.g. 100 rows in the flat file and always in the same format) that is eroding away any "well what about..." questions that would make the 16 months make sense. Does the scope of the work include anything beyond parsing the data and "sending it for processing"? E.g. you mention that after the parse you must "compare with a set of fields from your own invoice in SQL"... is this just something like string matching (make sure field A from the file and value for field A in my SQL table are identical) or is there something more to it (if the address fields don't match between flat file and the invoice in SQL then use Google Maps API to find best well-formatted address including zip+4)? Is there any piece of the "processing" (you mentioned "send it for processing") that has to be built in this scope of work as well? You see where I'm going I'm sure. That being said, if the scope is as simple as you describe it and you dev is a real dev (not someone from ops who claims they program in excel or whatever) then I have a hard time believing this would take someone more than a week or two.
__________________
Grant In the stable: 1938 Buick Special model 41, 1963 Solex 2200, 1973 Vespa Primavera 125, 1974 Vespa Rally 200, 1986 VW Vanagon Syncro Westfalia, 1989 VW Doka Tristar, 1995 Toyota Land Cruiser, 2011 Pursuit 315 OS, 2022 Tesla Y Gone but not forgotten: 1973 VW Beetle, 1989 Porsche 944, 2008 R56 Mini Cooper S |
||
|
03-10-2021, 03:26 PM
|
|
|
Registered
Join Date: Jul 2001
Location: Lawrenceville GA 30045
Posts: 7,415
|
For $50, you could probably go to the fiver web site and have this done overnight from the other side of the world, ready to use when you get your coffee in the morning.
__________________
Mark '83 SC Targa - since 5/5/2001 '06 911 S Aerokit - from 5/2/2016 to 11/14/2018 '11 911 S w/PDK - from 7/2/2021 to ??? |
||
|
03-10-2021, 03:43 PM
|
|
|
Monkey+Football
|
Quote:
16 months is too long to cry BS. Barring any procedural holdups, it should have been pretty obvious after 30 days something was forked.
__________________
<Insert witty comment> 85 Targa Wong Chip Fabspeed M&K Bilsteins and a bunch of other stuff. |
||
|
03-11-2021, 08:54 AM
|
|
|
Registered
|
Quote:
Sounds like the person parsing the file to do the compare is doing it correct, just the IT manager of the project didn't know how or what tools they were using. The issue is receiving the file and determining if it is the correct hash and has the correct fields and the team doing that is unable to do so as of yet. They don't know why the code rejects some files and not others. They don't know how to send the response back if good and they don't know how to send the response back to let the company sending the file what is missing. They do know it does reject some files.
__________________
Brent The X15 was the only aircraft I flew where I was glad the engine quit. - Milt Thompson. "Don't get so caught up in your right to dissent that you forget your obligation to contribute." Mrs. James to her son Chappie. |
||
|
03-11-2021, 09:32 AM
|
|
|
Registered
|
Just so you all know, this is still not live. They tried a couple weeks ago and forgot some stuff and lost 300,000 records. It didn't go well from there.
__________________
Brent The X15 was the only aircraft I flew where I was glad the engine quit. - Milt Thompson. "Don't get so caught up in your right to dissent that you forget your obligation to contribute." Mrs. James to her son Chappie. |
||
|
07-01-2021, 12:12 PM
|
|
|
|
Registered
Join Date: Mar 2003
Posts: 10,433
|
Quote:
|
||
|
07-01-2021, 04:44 PM
|
|
|
Driver, not Mechanic
Join Date: May 2013
Location: SF Bay Area
Posts: 3,024
|
The 90's called they want their FTP back.
|
||
|
07-01-2021, 08:30 PM
|
|
|
Registered
Join Date: Dec 1969
Location: chula vista ca usa
Posts: 5,724
|
As a retired Senior Oracle DBA I had to do something similar several times, if I read you original post correctly. Someone wants to give you some data so they generate a flat or "ASCII text" file for you. What I did was to see if there were matching fields in the various tables to match the data items in the incoming data file (text). I would generally create a single "temp" file to put the data into then generate a fairly simple query to compare the data. If the new data was good then it was inserted into their tables. Some things to check:
1. Are the data types matching? 2. if there are "char" or "varchar" data types are they 2 bit or 4 bit each. Some foreign data items are 4 bit characters! Hope it gets worked out as someone has made this much harder than it needs to be? John |
||
|
07-02-2021, 09:07 AM
|
|
|
Registered
|
Quote:
This isn't my project, but I know the PM on the production side, not the IT side. Wildthing, FTP would be a better option than what they are (not) doing right now. Honestly , I can't see people keeping their jobs after this. There is some background info I would love to give, but since this is a public thread I am hesitant to give some of that info out. Don't want to get the PM in trouble for me posting their internal struggles.
__________________
Brent The X15 was the only aircraft I flew where I was glad the engine quit. - Milt Thompson. "Don't get so caught up in your right to dissent that you forget your obligation to contribute." Mrs. James to her son Chappie. |
||
|
07-05-2021, 09:31 AM
|
|
|
Registered
|
John, it isn't even as complicated as you are describing.
__________________
Brent The X15 was the only aircraft I flew where I was glad the engine quit. - Milt Thompson. "Don't get so caught up in your right to dissent that you forget your obligation to contribute." Mrs. James to her son Chappie. |
||
|
07-05-2021, 09:31 AM
|
|
|
Get off my lawn!
|
It sounds like one of the projects my wife was involved in at the university from which she retired.
One group was trying to move data to one department from the state government. When they finally showed my wife what they did, they were proud of it. Then she pointed out several violations of the law as to sensitive personal data and HIPPA violations. The entire project vanished, and several people were fired, and they started over from scratch. Government money just wasted.
__________________
Glen 49 Year member of the Porsche Club of America 1985 911 Carrera; 2017 Macan 1986 El Camino with Fuel Injected 350 Crate Engine My Motto: I will never be too old to have a happy childhood! |
||
|
07-05-2021, 09:44 AM
|
|
|
|
Too big to fail
|
Quote:
__________________
"You go to the track with the Porsche you have, not the Porsche you wish you had." '03 E46 M3 '57 356A Various VWs |
||
|
07-05-2021, 11:34 AM
|
|
|
Registered
|
Quote:
It's their data, but our code Business owner - Recovery plan mode from the dev team NOW. Oh yeah, I'm retired, everyday is a Saturday
__________________
1980 911 - Metzger 3.6L 2016 Cayman S |
||
|
07-05-2021, 11:45 AM
|
|
|
Registered
|
Quote:
The plan from the devs was to do a manual fix for every record until they are up to speed. Sure, 3000k to 500k records per day. Should be no problem.
__________________
Brent The X15 was the only aircraft I flew where I was glad the engine quit. - Milt Thompson. "Don't get so caught up in your right to dissent that you forget your obligation to contribute." Mrs. James to her son Chappie. |
||
|
07-05-2021, 12:02 PM
|
|
Carrera
