lumberjack-developers
March 2013
lumberjack-developers@lists.fedorahosted.org
- 1 participants
- 2 discussions
need for a trusted subtree was: Re: [rsyslog] Parsing the CEE payload before the trusted properties?
by David Lang
cross-posting to the lumberjack mailing list where I raised the issue a few
months ago
This is a report from the field that demonstrates the problem that I was trying
to explain a few months ago on the lumberjack mailing list when I was advocating
for there to be a 'trusted' subtree that an admin could configure to say that
log input could never go there (instead being redirected to some other subtree)
Given that the trusted properties can be forged by root (and therefor sometimes
are not really trusted) and other times you want to accept trusted properties as
reported by a remote machine (where they could be forged by the sending
machine), this is not something that can be baked directly into rsyslog.
Instead this needs to be something under the control of the admin. As with any
controls, this needs to be made as simple as possible. Having to
whitelist/blacklist a bunch of differnet fields[1] is just bad idea.
Instead, there should be some subtree that the admin can define on a
case-by-case basis (I called it 'trusted') that all of these sorts of fields are
in. That way the admin can just define that subtree and anything under it gets
covered.
As I said in that discussion, there should be two subtrees that are defined with
this command, the 'trusted' one that is not accepted from the source, and the
'forged' one where any data that the source tries to send in either the
'trusted' or 'forged' subtrees gets moved to.
so you would remap
trusted!exe -> forged!exe
forged!exe -> forged!forged!exe
David Lang
[1] the list of what fields need protection is almost certinly going to change
over time and with different configurations.
Examples:
if namespaces are used, the values provided in the trusted properties may not
really be the right ones (or at least they won't match what the application
sees), so namespace information will probably be needed at some point
if the log is arriving from a remote system, fromhost-ip is a trusted
property, cert information from a TLS handshake is a trusted property, etc
On Fri, 15 Mar 2013, Philippe Muller wrote:
> Hello rsyslog-users,
>
> I really like the idea of imuxsock's trusted properties and structured
> logging.
>
> However, when using both features lets users override the trusted
> properties.
> I guess It's because the JSON parsing takes place after the trusted
> properties parsing.
>
> Here is my configuration :
> $ModLoad imuxsock
> $SystemLogUsePIDFromSystem on
> $SystemLogSocketAnnotate on
> $SystemLogParseTrusted on
> module(load="mmjsonparse")
> *.* :mmjsonparse:
> if $parsesuccess == "FAIL" then set $!msg = $msg;
> template(name="cee" type="string" string="%$!all-json%\n")
> action(type="omusrmsg" users="*" template="cee")
>
> Results:
> - logger foo
> => { "pid": 15385, "uid": 0, "gid": 0, "exe": "\/usr\/bin\/logger", "cmd":
> "logger foo ", "msg": " foo" }
> - logger '@cee: {"exe":"/bin/ls","cmd":"ls","pid":42,"gid":42,"uid":42}'
> => { "exe": "\/bin\/ls", "cmd": "ls", "pid": 42, "gid": 42, "uid": 42 }
>
> Is there a way to make mmjsonparse parses the CEE payload before parsing
> the trusted properties ? (making the trusted properties override user
> fields)
> Alternatively, is there a way to make mmjsonparse put user data in a
> subtree ?
11 years, 1 month
- 1
- 0
message truncation
by David Lang
no matter what you set the max message size to, if you allow user created fields
there is going to be some time when the log messages get too large to process.
The traditional syslog method is to truncate the message when you hit the limit
(originally 1k, but with modern syslogs, configurable by the admin)
However, when the message is a JSON structured log, what is the proper way to
deal with this problem, and how do the various syslog daemons handle this case?
I recently threw together a quick program to scrape java errors and send them to
syslog, I made it split the messages into 4k chunks (with message 2/4 type
tags), but I didn't test the case where a message didn't have a single space or
tab character in a 4k chunk and had the program go into a spin while trying to
send a production message that was like that.
David Lang
11 years, 2 months
- 1
- 0
- ← Newer
- 1
- Older →