Re: Parser Progress

Hi! This will be another annoyingly lengthy email from me :-). I worked on the parser and I'm finally getting some results. I have a demo implementation which can parse only some simple recipes. The center of it is XmlParser, which is a base class with the necessary helper functions, that handle proper resolving of included parts, processing aliases and template functions. The actual parser code can be split among multiple parser classes. In this example case, I used NetTestParse as the root recipe parser, MachineParse for parsing netmachineconfig and nettestconfig and CommandParse for parsing commands. All these parsers have a reference to recipe dictionary to which they store the values acquired during parsing - so the recipe is always up to date. XmlParser objects are connected into a sort-of-a tree. For instance NetTestParse is processing a XML and when it finds <machine> tag, it creates an instance of MachineParse and let's it take care for the whole tag. MachineParse could create some more subparsers if it wanted to, etc. Now, how exactly the parsing works. The code consists of element-handlers. Each element handler is a function/method with the following prototype: def _element_handler(node, params) where node is the DOM node object and params is a dictionary with any optional parameters. The root handler of each parser is the parse() method of XmlParser class. What a handler does is totally up to the programmer, but the usual actions are: 1. Process the element - check for attributes - acquire and check/convert attribute values - store the values to recipe dictionary - execute some actions tied to this particular element 2. Process child elements For processing child elements there's a predefined helper method in XmlParser called _process_child_elements(). def _process_child_elements(self, node, scheme) It takes two args, the current node and a scheme. Scheme is a dictionary mapping tag names to element-handler functions. The scheme defines what tags are expected and therefore allowed children of this tag. For instance handler function for <nettestrecipe> tag looks like this: def _nettestrecipe(self, node, params): scheme = {"machines": self._machines, "command_sequence": self._command_sequence} self._process_child_nodes(node, scheme) There is no processing tied to the tag itself, so the function only defines the scheme dict for children and processes them. Here is another example for the <info> tag: def _info(self, node, params): info = self._recipe["machines"][self._id]["info"] = {} info["hostname"] = self._get_attribute(node, "hostname") info["rootpass"] = self._get_attribute(node, "rootpass") Here we store some info directly into the recipe variable and we also don't go any further into the tree, because info tag should not have any children (in case it does, they're ignored). The XmlParser has some predefined handlers for <define> tag, so the later parsers doesn't need to worry about that at all. Another feature I added to the parser is the ability to report line+column numbers in error messages. So in case something goes wrong, you'll know exactly where it is. There's one question about the actions tied to the parsing. Should parser be able to send RPC requests to slaves or create interfaces in libvirt? I though about this and I think that our situation is comparable with syntax-driven compilation. Our parser needs to perform some additional checks (and requests) outside of its area (same as semantic checks that are performed by the parser in syntax-driven compilers). What do you think about this approach?

I like it because it keeps the current basic structure of the code, so we'll be able to reuse lots of the existing code from the current NetTestParser and NetConfigParser. It should also blend nicely to the other parts of the system as well (to NetConfig and NetTestController). Here's the full prototype implementation: https://url.corp.redhat.com/fd868a1 We definitely need to discuss this in person sometime when we all return from PTO, so I'm stopping the work on this until we discuss it.

Radek _______________________________________________ LNST-developers mailing list LNST-developers(a)lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/lnst-developers