[Secure Coding] master: Shell: Example code for recognizing non-negative integers (01cf74a)
by fweimer＠fedoraproject.org
Repository : http://git.fedorahosted.org/git/?p=secure-coding.git
On branch : master
>---------------------------------------------------------------
commit 01cf74aac034286d6dd3a341dcaab9cabcdb938f
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Fri Oct 10 16:19:15 2014 +0200
Shell: Example code for recognizing non-negative integers
>---------------------------------------------------------------
defensive-coding/en-US/Shell.xml | 40 ++++++++++++++++++++++++++++++++++++++
1 files changed, 40 insertions(+), 0 deletions(-)
diff --git a/defensive-coding/en-US/Shell.xml b/defensive-coding/en-US/Shell.xml
index 9c9ec1f..24554b1 100644
--- a/defensive-coding/en-US/Shell.xml
+++ b/defensive-coding/en-US/Shell.xml
@@ -203,6 +203,11 @@ external-program "$arg1" "$arg2"
linkend="sect-Defensive_Coding-Shell-Alternatives"/>.
</para>
</important>
+ <para>
+ If it is impossible to avoid shell arithmetic on untrusted
+ inputs, refer to <xref
+ linkend="sect-Defensive_Coding-Shell-Input_Validation"/>.
+ </para>
</section>
<section id="sect-Defensive_Coding-Shell-Types">
<title>Type declarations</title>
@@ -376,6 +381,41 @@ trap cleanup 0
</informalexample>
</example>
</section>
+<section id="sect-Defensive_Coding-Shell-Input_Validation">
+ <title>Performing input validation</title>
+ <para>
+ In some cases, input validation cannot be avoided. For example,
+ if arithmetic evaluation is absolutely required, it is imperative
+ to check that input values are, in fact, integers. See <xref
+ linkend="sect-Defensive_Coding-Shell-Arithmetic"/>.
+ </para>
+ <para>
+ The following construct can be used to check if a string
+ “<literal>$value</literal>” is not a non-negative integer.
+ </para>
+ <informalexample>
+ <programlisting language="Bash">
+case "$value" in
+ *[!0-9]*)
+ echo "invalid input value" 1>&2
+ exit 1
+ ;;
+esac
+ </programlisting>
+ </informalexample>
+ <para>
+ The pattern “<literal>*[!0-9]*</literal>” is not special shell
+ syntax—it matches any string which contains arbitrary characters,
+ followed by a non-digit, followed by arbitrary characters.
+ </para>
+ <para>
+ Using <literal>case</literal> statements is the most reliable way
+ for performing input validation, although constructing proper
+ patterns is difficult. The <literal>expr</literal> external
+ command and the built-in operator <literal>=~</literal> can give
+ misleading results.
+ </para>
+</section>
<section id="sect-Defensive_Coding-Shell-Edit_Guard">
<title>Guarding shell scripts against changes</title>
<para>
9 years, 6 months
- 1
- 0
[Secure Coding] master: Add a chapter on shell programming (2a82911)
by fweimer＠fedoraproject.org
Repository : http://git.fedorahosted.org/git/?p=secure-coding.git
On branch : master
>---------------------------------------------------------------
commit 2a829115ffd64e3ce6c6a281498f8a0006a77af3
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Fri Oct 10 15:36:28 2014 +0200
Add a chapter on shell programming
>---------------------------------------------------------------
defensive-coding/en-US/Defensive_Coding.xml | 1 +
defensive-coding/en-US/Shell.xml | 409 +++++++++++++++++++++++++++
2 files changed, 410 insertions(+), 0 deletions(-)
diff --git a/defensive-coding/en-US/Defensive_Coding.xml b/defensive-coding/en-US/Defensive_Coding.xml
index ee96c8d..58d9121 100644
--- a/defensive-coding/en-US/Defensive_Coding.xml
+++ b/defensive-coding/en-US/Defensive_Coding.xml
@@ -8,6 +8,7 @@
<xi:include href="CXX.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
<xi:include href="Java.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
<xi:include href="Python.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
+ <xi:include href="Shell.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
<xi:include href="Go.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
<xi:include href="Vala.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
</part>
diff --git a/defensive-coding/en-US/Shell.xml b/defensive-coding/en-US/Shell.xml
new file mode 100644
index 0000000..9c9ec1f
--- /dev/null
+++ b/defensive-coding/en-US/Shell.xml
@@ -0,0 +1,409 @@
+<?xml version='1.0' encoding='utf-8' ?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+]>
+<chapter id="chap-Defensive_Coding-Shell">
+<title>Shell Programming and <application>bash</application></title>
+<para>
+ This chapter contains advice about shell programming, specifically
+ in <application>bash</application>. Most of the advice will apply
+ to scripts written for other shells because extensions such as
+ integer or array variables have been implemented there as well, with
+ comparable syntax.
+</para>
+<section id="sect-Defensive_Coding-Shell-Alternatives">
+ <title>Consider alternatives</title>
+ <para>
+ Once a shell script is so complex that advice in this chapter
+ applies, it is time to step back and consider the question: Is
+ there a more suitable implementation language available?
+ </para>
+ <para>
+ For example, Python with its <literal>subprocess</literal> module
+ can be used to write scripts which are almost as concise as shell
+ scripts when it comes to invoking external programs, and Python
+ offers richer data structures, with less arcane syntax and more
+ consistent behavior.
+ </para>
+</section>
+<section id="sect-Defensive_Coding-Shell-Language">
+<title>Shell language features</title>
+<para>
+ The following sections cover subtleties concerning the shell
+ programming languages. They have been written with the
+ <application>bash</application> shell in mind, but some of these
+ features apply to other shells as well.
+</para>
+<para>
+ Some of the features described may seem like implementation defects,
+ but these features have been replicated across multiple independent
+ implementations, so they now have to be considered part of the shell
+ programming language.
+</para>
+<section id="sect-Defensive_Coding-Shell-Parameter_Expansion">
+ <title>Parameter expansion</title>
+ <para>
+ The mechanism by which named shell variables and parameters are
+ expanded is called <emphasis>parameter expansion</emphasis>. The
+ most basic syntax is
+ “<literal>$</literal><emphasis>variable</emphasis>” or
+ “<literal>${</literal><emphasis>variable</emphasis><literal>}</literal>”.
+ </para>
+ <para>
+ In almost all cases, a parameter expansion should be enclosed in
+ double quotation marks <literal>"</literal>…<literal>"</literal>.
+ </para>
+ <informalexample>
+ <programlisting language="Bash">
+external-program "$arg1" "$arg2"
+ </programlisting>
+ </informalexample>
+ <para>
+ If the double quotation marks are omitted, the value of the
+ variable will be split according to the current value of the
+ <envar>IFS</envar> variable. This may allow the injection of
+ additional options which are then processed by
+ <literal>external-program</literal>.
+ </para>
+ <para>
+ Parameter expansion can use special syntax for specific features,
+ such as substituting defaults or performing string or array
+ operations. These constructs should not be used because they can
+ trigger arithmetic evaluation, which can result in code execution.
+ See <xref linkend="sect-Defensive_Coding-Shell-Arithmetic"/>.
+ </para>
+</section>
+<section id="sect-Defensive_Coding-Shell-Double_Expansion">
+ <title>Double expansion</title>
+ <para>
+ <emphasis>Double expansion</emphasis> occurs when, during the
+ expansion of a shell variable, not just the variable is expanded,
+ replacing it by its value, but the <emphasis>value</emphasis> of
+ the variable is itself is expanded as well. This can trigger
+ arbitrary code execution, unless the value of the variable is
+ verified against a restrictive pattern.
+ </para>
+ <para>
+ The evaluation process is in fact recursive, so a self-referential
+ expression can cause an out-of-memory condition and a shell crash.
+ </para>
+ <para>
+ Double expansion may seem like as a defect, but it is implemented
+ by many shells, and has to be considered an integral part of the
+ shell programming language. However, it does make writing robust
+ shell scripts difficult.
+ </para>
+ <para>
+ Double evaluation can be requested explicitly with the
+ <literal>eval</literal> built-in command, or by invoking a
+ subshell with “<literal>bash -c</literal>”. These constructs
+ should not be used.
+ </para>
+ <para>
+ The following sections give examples of places where implicit
+ double expansion occurs.
+ </para>
+ <section id="sect-Defensive_Coding-Shell-Arithmetic">
+ <title>Arithmetic evaluation</title>
+ <para>
+ <emphasis>Arithmetic evaluation</emphasis> is a process by which
+ the shell computes the integer value of an expression specified
+ as a string. It is highly problematic for two reasons: It
+ triggers double evaluation (see <xref
+ linkend="sect-Defensive_Coding-Shell-Arithmetic"/>), and the
+ language of arithmetic expressions is not self-contained. Some
+ constructs in arithmetic expressions (notably array subscripts)
+ provide a trapdoor from the restricted language of arithmetic
+ expressions to the full shell language, thus paving the way
+ towards arbitrary code execution. Due to double expansion,
+ input which is (indirectly) referenced from an arithmetic
+ expression can trigger execution of arbitrary code, which is
+ potentially harmful.
+ </para>
+ <para>
+ Arithmetic evaluation is triggered by the follow constructs:
+ </para>
+ <!-- The list was constructed by looking at the bash sources and
+ search for the string "expand_". -->
+ <itemizedlist>
+ <listitem>
+ <para>
+ The <emphasis>expression</emphasis> in
+ “<literal>$((</literal><emphasis>expression</emphasis><literal>))</literal>”
+ is evaluated. This construct is called <emphasis>arithmetic
+ expansion</emphasis>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ “<literal>$[</literal><emphasis>expression</emphasis><literal>]</literal>”
+ is a deprecated syntax with the same effect.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The arguments to the <literal>let</literal> shell built-in
+ are evaluated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ “<literal>((</literal><emphasis>expression</emphasis><literal>))</literal>”
+ is an alternative syntax for “<literal>let
+ </literal><emphasis>expression</emphasis>”.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Conditional expressions surrounded by
+ “<literal>[[</literal>…<literal>]]</literal>” can trigger
+ arithmetic evaluation if certain operators such as
+ <literal>-eq</literal> are used. (The
+ <literal>test</literal> built-in does not perform arithmetic
+ evaluation, even with integer operators such as
+ <literal>-eq</literal>.)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Certain parameter expansions, for example
+ “<literal>${</literal><emphasis>variable</emphasis><literal>[</literal><emphasis>expression</emphasis><literal>]}</literal>”
+ (array indexing) or
+ “<literal>${</literal><emphasis>variable</emphasis><literal>:</literal><emphasis>expression</emphasis><literal>}</literal>”
+ (string slicing), trigger arithmetic evaluation of
+ <emphasis>expression</emphasis>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Assignment to array elements using
+ “<emphasis>array_variable</emphasis><literal>[</literal><emphasis>subscript</emphasis><literal>]=</literal><emphasis>expression</emphasis>”
+ triggers evaluation of <emphasis>subscript</emphasis>, but
+ not <emphasis>expression</emphasis>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The expressions in the arithmetic <literal>for</literal>
+ command,
+ “<literal>for ((</literal><emphasis>expression1</emphasis><literal>; </literal><emphasis>expression2</emphasis><literal>; </literal><emphasis>expression3</emphasis><literal>)); do </literal><emphasis>commands</emphasis><literal>; done</literal>”
+ are evaluated. This does not apply to the regular
+ for command,
+ “<literal>for </literal><emphasis>variable</emphasis><literal> in </literal><emphasis>list</emphasis><literal>; do </literal><emphasis>commands</emphasis><literal>; done</literal>”.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <important>
+ <para>
+ Depending on the <application>bash</application> version, the
+ above list may be incomplete.
+ </para>
+ <para>
+ If faced with a situation where using such shell features
+ appears necessary, see <xref
+ linkend="sect-Defensive_Coding-Shell-Alternatives"/>.
+ </para>
+ </important>
+ </section>
+ <section id="sect-Defensive_Coding-Shell-Types">
+ <title>Type declarations</title>
+ <para>
+ <application>bash</application> supports explicit type
+ declarations for shell variables:
+ </para>
+ <informalexample>
+ <programlisting language="Bash">
+ declare -i integer_variable
+ declare -a array_variable
+ declare -A assoc_array_variable
+
+ typeset -i integer_variable
+ typeset -a array_variable
+ typeset -A assoc_array_variable
+
+ local -i integer_variable
+ local -a array_variable
+ local -A assoc_array_variable
+
+ readonly -i integer_variable
+ readonly -a array_variable
+ readonly -A assoc_array_variable
+ </programlisting>
+ </informalexample>
+ <para>
+ Variables can also be declared as arrays by assigning them an
+ array expression, as in:
+ </para>
+ <informalexample>
+ <programlisting language="Bash">
+array_variable=(1 2 3 4)
+ </programlisting>
+ </informalexample>
+ <para>
+ Some built-ins (such as <literal>mapfile</literal>) can
+ implicitly create array variables.
+ </para>
+ <para>
+ Such type declarations should not be used because assignment to
+ such variables (independent of the concrete syntax used for the
+ assignment) triggers arithmetic expansion (and thus double
+ expansion) of the right-hand side of the assignment operation.
+ See <xref linkend="sect-Defensive_Coding-Shell-Arithmetic"/>.
+ </para>
+ <para>
+ Shell scripts which use integer or array variables should be
+ rewritten in another, more suitable language. Se <xref
+ linkend="sect-Defensive_Coding-Shell-Alternatives"/>.
+ </para>
+ </section>
+</section>
+<section id="sect-Defensive_Coding-Shell-Obscure">
+ <title>Other obscurities</title>
+ <para>
+ Obscure shell language features should not be used. Examples are:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Exported functions (<literal>export -f</literal> or
+ <literal>declare -f</literal>).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Function names which are not valid variable names, such as
+ “<literal>module::function</literal>”.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The possibility to override built-ins or external commands
+ with shell functions.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Changing the value of the <envar>IFS</envar> variable to
+ tokenize strings.
+ </para>
+ </listitem>
+ </itemizedlist>
+</section>
+</section>
+<section id="sect-Defensive_Coding-Shell-Invoke">
+<title>Invoking external commands</title>
+<para>
+ When passing shell variables as single command line arguments,
+ they should always be surrounded by double quotes. See
+ <xref linkend="sect-Defensive_Coding-Shell-Parameter_Expansion"/>.
+</para>
+<para>
+ Care is required when passing untrusted values as positional
+ parameters to external commands. If the value starts with a hyphen
+ “<literal>-</literal>”, it may be interpreted by the external
+ command as an option. Depending on the external program, a
+ “<literal>--</literal>” argument stops option processing and treats
+ all following arguments as positional parameters. (Double quotes
+ are completely invisible to the command being invoked, so they do
+ not prevent variable values from being interpreted as options.)
+</para>
+<para>
+ Cleaning the environment before invoking child processes is
+ difficult to implement in script. <application>bash</application>
+ keeps a hidden list of environment variables which do not correspond
+ to shell variables, and unsetting them from within a
+ <application>bash</application> script is not possible. To reset
+ the environment, a script can re-run itself under the “<literal>env
+ -i</literal>” command with an additional parameter which indicates
+ the environment has been cleared and suppresses a further
+ self-execution. Alternatively, individual commands can be executed
+ with “<literal>env -i</literal>”.
+</para>
+<important>
+ <para>
+ Completely isolation from its original execution environment
+ (which is required when the script is executed after a trust
+ transition, e.g., triggered by the SUID mechanism) is impossible
+ to achieve from within the shell script itself. Instead, the
+ invoking process has to clear the process environment (except for
+ few trusted variables) before running the shell script.
+ </para>
+</important>
+<para>
+ Checking for failures in executed external commands is recommended.
+ If no elaborate error recovery is needed, invoking “<literal>set
+ -e</literal>” may be sufficient. This causes the script to stop on
+ the first failed command. However, failures in pipes
+ (“<literal>command1 | command2</literal>”) are only detected for the
+ last command in the pipe, errors in previous commands are ignored.
+ This can be changed by invoking “<literal>set -o pipefail</literal>”.
+ Due to architectural limitations, only the process that spawned
+ the entire pipe can check for failures in individual commands;
+ it is not possible for a process to tell if the process feeding
+ data (or the process consuming data) exited normally or with
+ an error.
+</para>
+<para>
+ See <xref linkend="sect-Defensive_Coding-Tasks-Processes-Creation"/>
+ for additional details on creating child processes.
+</para>
+</section>
+<section id="sect-Defensive_Coding-Shell-Temporary_Files">
+ <title>Temporary files</title>
+ <para>
+ Temporary files should be created with the
+ <literal>mktemp</literal> command, and temporary directories with
+ “<literal>mktemp -d</literal>”.
+ </para>
+ <para>
+ To clean up temporary files and directories, write a clean-up
+ shell function and register it as a trap handler, as shown in
+ <xref linkend="ex-Defensive_Coding-Tasks-Temporary_Files"/>.
+ Using a separate function avoids issues with proper quoting of
+ variables.
+ </para>
+ <example id="ex-Defensive_Coding-Tasks-Temporary_Files">
+ <title>Creating and cleaning up temporary files</title>
+ <informalexample>
+ <programlisting language="Bash">
+tmpfile="$(mktemp)"
+
+cleanup () {
+ rm -f -- "$tmpfile"
+}
+
+trap cleanup 0
+ </programlisting>
+ </informalexample>
+ </example>
+</section>
+<section id="sect-Defensive_Coding-Shell-Edit_Guard">
+ <title>Guarding shell scripts against changes</title>
+ <para>
+ <application>bash</application> only reads a shell script up to
+ the point it is needed for executed the next command. This means
+ that if script is overwritten while it is running, execution can
+ jump to a random part of the script, depending on what is modified
+ in the script and how the file offsets change as a result. (This
+ behavior is needed to support self-extracting shell archives whose
+ script part is followed by a stream of bytes which does not follow
+ the shell language syntax.)
+ </para>
+ <para>
+ Therefore, long-running scripts should be guarded against
+ concurrent modification by putting as much of the program logic
+ into a <literal>main</literal> function, and invoking the
+ <literal>main</literal> function at the end of the script, using
+ this syntax:
+ </para>
+ <informalexample>
+ <programlisting language="Bash">
+main "$@" ; exit $?
+ </programlisting>
+ </informalexample>
+ <para>
+ This construct ensures that <application>bash</application> will
+ stop execution after the <literal>main</literal> function, instead
+ of opening the script file and trying to read more commands.
+ </para>
+</section>
+</chapter>
9 years, 6 months
- 1
- 0
[Secure Coding] master: Go: Fix typo (fab2049)
by fweimer＠fedoraproject.org
Repository : http://git.fedorahosted.org/git/?p=secure-coding.git
On branch : master
>---------------------------------------------------------------
commit fab2049127ec0a116a15d1eff478ce14b31d15af
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Tue Oct 7 14:16:27 2014 +0200
Go: Fix typo
>---------------------------------------------------------------
defensive-coding/en-US/Go.xml | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/defensive-coding/en-US/Go.xml b/defensive-coding/en-US/Go.xml
index e1285ed..06098e7 100644
--- a/defensive-coding/en-US/Go.xml
+++ b/defensive-coding/en-US/Go.xml
@@ -22,7 +22,7 @@
Code which does not run in parallel and does not use the
<literal>unsafe</literal> package (or other packages which expose
unsafe constructs) is memory-safe. For example, invalid casts and
- out-of-range subscripting cause panics and run time.
+ out-of-range subscripting cause panics at run time.
</para>
<para>
Keep in mind that finalization can introduce parallelism because
9 years, 6 months
- 1
- 0
[Secure Coding] master: Go Marshaling: Fix section title (00ae7f5)
by fweimer＠fedoraproject.org
Repository : http://git.fedorahosted.org/git/?p=secure-coding.git
On branch : master
>---------------------------------------------------------------
commit 00ae7f52e51702cdc856cd60ff1a34f4f5046c87
Author: Florian Weimer <fweimer(a)redhat.com>
Date: Thu Aug 14 10:03:03 2014 +0200
Go Marshaling: Fix section title
>---------------------------------------------------------------
defensive-coding/en-US/Go.xml | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/defensive-coding/en-US/Go.xml b/defensive-coding/en-US/Go.xml
index b5529a6..e1285ed 100644
--- a/defensive-coding/en-US/Go.xml
+++ b/defensive-coding/en-US/Go.xml
@@ -88,7 +88,7 @@
</para>
</section>
<section id="chap-Defensive_Coding-Go-Marshaling">
- <title>Marshaling and marshaling</title>
+ <title>Marshaling and unmarshaling</title>
<para>
Several packages in the <literal>encoding</literal> hierarchy
provide support for serialization and deserialization. The usual
9 years, 6 months
- 1
- 0
Fedora Activity Day - 1st Nov 2014 - theme security
by P J P
Hello all,
See -> https://fedoraproject.org/wiki/FAD_Pune_Security_1
Date: Say, 1st Nov 2014
Venue: Red Hat Inc. Tower-10, Magarpatta City, Near Hadapsar, Pune, India.
On 1st Nov 2014, we plan to host a Fedora Activity Day(FAD) geared towards triaging security bugs in Fedora. The day would start with a brief introduction to Fedora Security and progress towards collective bug triaging.
If you are in Pune or plan to be here on 1st Nov, please feel free to drop in and join the action. :)
Please note:- we have limited capacity(=~25) to accommodate participants.
...see you there! :)
---
Regards
-Prasad
http://feedmug.com
9 years, 6 months
- 1
- 0