Support batch operation

[petri]

It would be useful if one could convert a whole bunch exported CSV files in one shot; optionally combining the parsed statements/entries into a single OFX file.

[kedder]

Converting batch of csvs is simple enough with shell script, I think, like so:

for F in $FILES; do ofxstatement convert -t <...> $F $F.ofx ; done

Combining multiple statements into one might be tricky, because statements include "start balance" and "end balance", and their difference is corresponds with sum of all transactions, so if part of the date period is missing, that might not be true.

In my case, I just generate statement csv for larger period from my bank and convert it in one go. What is your use case?

[petri]

Unfortunately, the bank here does not provide CSV exports for an arbitrary larger period; the CSV always comes in batches (of one full month, from first of month to last). They don't include a balance either - just the transactions. Combining raw CSV files could be a workaround; just need to strip off the first (header) lines.

The batch operation would just be an useful convenience, for anyone wanting to start using something that can import OFX. For example, I have CSVs starting from 2010.

FWIW, here's what I put together for batch conversion & HTML generation:

#!/bin/bash

for FILEPATH in $1/*
  do
     parts=(${FILEPATH//\// })
     filename=${parts[5]}
     splitsuffix=(${filename//./ })
     nosuffix=${splitsuffix[0]}
     ./bin/ofxstatement convert -t osuuspankki $FILEPATH /tmp/$nosuffix.ofx
     echo "Converted $filename"
     xsltproc transform.xslt /tmp/$nosuffix.ofx > /tmp/$nosuffix.html
     echo "Generated HTML"
 done

(using the transform from https://gist.github.com/petri/7300330)

[kedder]

Hm, I don't think I want to complicate things that much. If it is possible to preprocess your csvs and combine them into one with external tools, I think it is a way to go. You can even filter out additional CSV headers in a parser itself, so you could just say

cat csvs/* > combined.csv
ofxstatement convert -t osuuspankki combined.csv combined.ofx

Another idea: make ofxstatement to work as a filter, i.e. read from stdin and write to stdout in case input and output filenames are not provided. This way you can feed it as many files as you want, provided that parser will digest the stream.

[petri]

Thanks for your suggestions. There have been a number of variations in the CSV files through the years, so preprocessing CSVs would repeat much of what's in the plugins already. Having to handle an aggregate file that possibly mixes all those variations would also be an additional piece of functionality.

I can make do with the bash script doing the batching. But I like the idea of ofxstatement being able to work as a filter; that was a good one, making it easy for new users to run all their CSVs through ofxstatement in a single command-line rather than having to write a full-blown bash script. That's quite nice.