New plugin API

Hi all, First of all thank you for working on Fedora-Review, which was really useful for me -- saved me a lot of time and probably improved quality of my reviews. I would like to share my opinion on how Fedora-Review plugin API should look like from perspective of a user who did a number of reviews and who doesn't know the internal API. PREAMBLE Normal F-R workflow (simplified a lot): First you build the package and run a number of automated tests (as part of F-R and external tools). Then go through the review template and perform manual checks. In most cases you can't open and read every source file, but you you'll probably end up running tools like find and grep looking for some package-specific patterns. After doing your n-th review you realize which parts are boring and brainless -- you find candidates for test automation. At this point you could probably write a custom script or a F-R plugin (using Python or JSON API). Writing a custom plugin requires a lot of time (I know from my own experience), so I think that the first option is definitely better. After all you remember well all the commands you had to run, you just need to write them down in a form of a script. You can even copy variuos parts directly from your shell history. But running these scripts manually again and again can be annoying. What you need is a glue between F-R and the scripts -- the API I am proposing. SOME ASSUMPTIONS The API should: 1) Be easy to learn. If you need to spend several (or more) hours to learn the F-R API and API of JSON some JSON library then creating a custom test (plugin) may be not worth the effort. However if you somehoe know most of the API before you even start thinking about plugins (more about that below), creating test scripts will be straightforward. 2) Be multilangual. You may want to write the simplest tests in just Bourne shell, maybe more complicated ones in other scripting languages. Also Perl SIG may want to write tests in Perl, Python in Python, Java SIG in... anything but Java :) You can't expect Perl proponents to write Python code only to create a plugin for F-R. 3) Backwards compatible. If you spend some time developing a set of plugins you don't want to rewrite them because of F-R upgrade. During upgrade from 0.1.x to 0.2.x I had to rewrite my plugin. Hopefully I had only one and Stano helped me with the upgrade. 4) Callback-less. Current JSON API gives plugins only some information about package being reviewed and requires them to call F-R back to obtain more info (like sections of spec file). This means formating a request, reading and parsing (semi-manually!) the reply. But this information could have been automatically provided by F-R from the beginning! F-R should give pluging all info it has about the package without plugins having to ask for it. MY PROPOSAL Basic information about packages, like name, version, type (C,Java,R,...) is passed in exported environmental variables, which names should be compatible with names from `rpm --querytags` if possible. More detailed information is passed on disk in form of files. This includes things like spec file, unpacked sources, rpmlint output, symlinks to builddir, unpacked binary rpms and so on. Users already should know basic RPM macros like %{name} or %{epoch}, so using env variables ($name and $epoch respectively) should be strightforward. People that previously used F-R will be familiar with structure of directories created by F-R (srpm-unpacked, upstream, BUILD, and so on). Test results are simply printed on stdout. Exit code decides if test passed, failed or was inconclusive (exit codes for that are already standardized). Attachements can be created by creating attachement files somewhere in the filesystem. I believe that implementing something similar to my approach would greatly enchance Fedora Review. It could be implemented as a JSON plugin (what I actually did) which then has a separate api for comunicating with different, scripts or implemented directly in Python. It's up to you, Fedora-Review developers, to decide if you want to support my idea or not. If yes then I would be glad to help. In any case I have my implementation working now so I'm happy either way :) Keep up with good work! Mikolaj Izdebski