[ale] coding practices

Wed Mar 3 18:31:11 EST 2010

You are thinking along the same lines I am. The app can be modified but that
would require negotiations/updates/money/work. I have a watchdog to do the
deed already (mostly) written. done files is named filename.done so it
matches up with the naming scheme of the source data file. While we have a
support agreement, I've hit several situations where what I needed to do
would alter the program and affect the support agreement. Thus I had to stop
pursuing that line.

Given that we can can not change the app without support issues, changing
the output seems to be a bad idea. I like splitting the metadata to a
separate file anyway. It makes the check easier to do:

if [ $(wc -l filename) -eq $(tail -n 1 filename.done) ]; then ...

The done file is a requirement of the third party group as part of their
practices.

On Wed, Mar 3, 2010 at 5:59 PM, scott <scott at sboss.net> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> a few questions that will help you in the right direction (whatever it may
> be):
>
> *) the non-free app (w/ database), do you a support contract with the
> company?
>
>        if so, have you asked them if that is an option for them to add the
> rowcount to the end of the output file?
>
>        Have you asked them if you add the rowcount to the end of the file,
> will it break their application?
>
> *) instead of modifing the non-free app (w/ database) to create a done file
> for the third party app, can you write a watchdog script/daemon (in the
> language of your choice) to monitor the output file and when it is
> completely there, to create the done file.  For instance to a md5sum (or
> some checksum hash of your choice) on the file, and the next time you check
> the file to see if the md5sum is the same.  If so, touch the done file
> otherwise update the "internal registry" for the new md5sum of the file.
>  repeat  This is good way since you are not changing any commercial/non-free
> apps.  It is some duct tape around the two apps so they work together
> better.
>
> *) you comment about putting the rowcount in the done file.   That is not a
> bad idea.  Another thing is you can put the md5sum of the file too. Well
> there is lots you can do but time/effort is your limiters.
>
>        Did the third party app say why they want the rowcount in the output
> file over a secondary file?
>
>        How about if ${filename} is the filename of the output file, why not
> ${filename}.done for the done file and ${filename}.md5sum for the md5sum?
>  Yeah you are creating extra files but they are small and you can use
> scripts/programs to use them to verify the files for the commercial apps.
>
>
> Here is my personal opinion, and please take it with a large grain of salt
> (size of a salt lick would be good!).  Dont change the input/output files of
> a commercial application.  Now if you can request the vendor to make the
> mods for you, that is great.  But you dont want to get in a situation where
> you are in a situation where the process is broken and the vendor blames you
> modifying the input/output files.  Depending on the vendor and your
> relationship with them depends on how much further at that point they will
> help you.  Most vendors have some sort of RFE (Request For Enhancement -
> they might call it something else) process.  Generally if I have seen if
> more than one customer asks for something most of them will add it, even if
> it is an option.
>
> It also might be good to get someone(s) from each of the vendor companies
> in the same room (or virtual room) to discuss what you are trying to do and
> let the two vendors duke it out over put what in what file and so forth.
>  Sometimes that is helpful to move the process along instead of you playing
> messenger between them.
>
> Good Luck!!
>
>
> On Mar 3, 2010, at 5:31 PM, Jim Kinney wrote:
>
> > I am having a disagreement with a procedure and am seeking other views.
> >
> > A non-free app generates a data file from a database function (database
> also part of the non-free app).
> >
> > A third party application needs to scp that file elsewhere for further
> processing.
> >
> > The third party wants a "done" file to indicate the app has finished
> writing to the file. That's not needed due to how the app does it's write
> then a move to the final location. No problem to touch a done file.
> >
> > Third party also wants the total row count in the data file appended to
> the data file.
> >
> > This is where I disagree. I would far rather put the additional data in
> the done file and not alter the output in anyway.
> >
> > Granted, adding the row count is trivial (wc -l filename >> filename) and
> that last line will be nothing like the actual data lines. It does make
> reprocessing the data files more complicated as they have to be checked for
> the presence of the row count on the last line before rerunning the import
> process again.
> >
> > Other views?
> >
> > --
> > --
> > James P. Kinney III
> > Actively in pursuit of Life, Liberty and Happiness
> >
> > _______________________________________________
> > Ale mailing list
> > Ale at ale.org
> > http://mail.ale.org/mailman/listinfo/ale
> > See JOBS, ANNOUNCE and SCHOOLS lists at
> > http://mail.ale.org/mailman/listinfo
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.12 (Darwin)
>
> iQIcBAEBCgAGBQJLjunXAAoJEF51inK7SsNsWnMQAJAJGoof/jSrQNPpMMLBKDwn
> d8jIs5qwI2bGxR739sJNhJtrjLynoeEWDVDiHVeHowsSvwKH5QDC9mT77fSutyKA
> nH08Z7Pk5/B7xsqlkx/rwYQkt1Nua97mHmh/qa32NSLxQN5U+hQshUv2gDXTQwiN
> w+hDcYbZVLINdkYbZ1BoR1g2q/bE+wcqHC3x45VM1iJgz9jlM/9JjbhJZASmxIHp
> 56Aeets8fz19luPJAE7n7gX1/odVsGOpvC0wbcFchejAyKpTX1lj4ww4wVB9uQHh
> gXL3d2gZPEOnuqoSU82iO3KqzVaVVG/1yapcgubKgl2P2qZqu17VdgkyvgnsXWLZ
> yiRK/+m6WIvHAWRmSQbDGMNUIOrYPHDGPJhDmV4pMcHutH71E0jiyd4uDiCpvhLN
> o4znh5VujoywvnYz2bmw6XwnMDsxGGU6WN4Wwqxa1YsoeUaoYPpjWvUv5ykOlIy/
> kYbt1Zy8drp2lNwgaFSEjyiVOtZqKqNKh92yd43nnu6R/vTfSHxxdncEX8LhnqIi
> QlWF5G0IOJNgihNh4Ub4KJ2ElyURpIZczkOM9VFoLaRC4buTLqmJlKj1rTy2mXiI
> bz4uwnyUhXJoTHFm866e5UIBMs95hgu00j8S/coprW6fG4jRtlifx50bnBle2tMk
> Mjr1Sme+R1BDo5A/LM+b
> =uVvE
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>

-- 
-- 
James P. Kinney III
Actively in pursuit of Life, Liberty and Happiness
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.ale.org/pipermail/ale/attachments/20100303/b2116c72/attachment-0001.html 