Commit graph

624 commits

Author SHA1 Message Date
Ronald Cron 27f84fc75c
Merge pull request #4813 from JoeSubbiani/TranslateCiphersuite_dev
Translate ciphersuite names
2021-09-27 08:57:52 +02:00
Jerry Yu 7a5ab044ca Add tls13 test with everst and ecp restartable
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
2021-09-15 22:06:11 +08:00
Joe Subbiani a25ffab422 Integrate tests as unit tests into one file
Rather than having the tests seperated into different files, they were integrated
into translate_ciphers.py and can be run from root using:
`python -m unittest tests/scripts/translate_ciphers.py`

test_translate_ciphers_format.sh was originally made as a testing ground before
having the translation tool being implmented into compat.sh. Translating it to
python code makes it redundant and therefore it will be removed.

Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
2021-09-03 13:19:50 +01:00
Joe Subbiani d614c0b197 Include translate ciphers tests in all.sh
To run test_translate_ciphers_names.py and _format.sh in the CI, include
it in all.sh component_check_generate_test_code.

Rename check_generate_test_code to check_test_helpers

Signed-off-by: Joe Subbiani <joe.subbiani@arm.com>
2021-09-03 13:18:50 +01:00
Gilles Peskine 0bf740ee4f
Merge pull request #4765 from gilles-peskine-arm/all.sh-subshells-3.0
Run all.sh components in a subshell
2021-09-02 10:26:58 +02:00
Manuel Pégourié-Gonnard e45ee40f7e
Merge pull request #4811 from hanno-arm/tls13_ciphersuite_api
Add TLS 1.3 ciphersuite and key exchange identifiers and API
2021-08-30 09:47:46 +02:00
Manuel Pégourié-Gonnard 4512f21473
Merge pull request #3572 from mpg/add-arm-linux-build
Add arm-linux-gnueabi-gcc build
2021-08-12 13:16:02 +02:00
Hanno Becker ae336852c5 Add ssl-opt.sh run to TLS 1.3 test in all.sh
Signed-off-by: Hanno Becker <hanno.becker@arm.com>
2021-08-12 06:28:45 +01:00
Gilles Peskine 80ddb991c2 Add --restore option to clean up but not necessarily run components
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-06 11:51:59 +02:00
Gilles Peskine 03af678911 Documentation improvements
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-06 11:35:17 +02:00
Gilles Peskine 86f6129067 Documentation improvement
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-05 15:11:33 +02:00
Gilles Peskine 7530163f3b Make --quiet more effective when running make generated_files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-05 15:10:47 +02:00
Gilles Peskine bf66e2cc8f Documentation improvements
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-03 13:44:28 +02:00
Gilles Peskine 1d475b6398 Disable wildcards when checking for unsupported components
Otherwise $COMMAND_LINE_COMPONENTS would try to expand wildcard patterns
based on files in the current directory.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-03 13:43:36 +02:00
Manuel Pégourié-Gonnard ae505eeeed Fix missing dependency on Travis
Was getting errors like:

In file included from /usr/include/limits.h:25:0,
                 from /usr/lib/gcc-cross/arm-linux-gnueabi/5/include-fixed/limits.h:168,
                 from /usr/lib/gcc-cross/arm-linux-gnueabi/5/include-fixed/syslimits.h:7,
                 from /usr/lib/gcc-cross/arm-linux-gnueabi/5/include-fixed/limits.h:34,
                 from ../include/mbedtls/check_config.h:30,
                 from ../include/mbedtls/build_info.h:81,
                 from common.h:26,
                 from asn1write.c:20:
/usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or directory

There are two packages to choose from: armhf or armel. Since the comment
in all.sh says we're trying to be close to Debian's "armel"
architecture, choose that, and fix a comment that was mentioning
gnueabihf for no apparent reason.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-08-03 11:19:59 +02:00
Manuel Pégourié-Gonnard 3a6c76937a Add arm-linux-gnueabi-gcc build to all.sh
Currently it can't be mandatory, since we can't install the required toolchain
on Jenkins right away.

Also, while at it, remove `SHELL='sh -x'` from the other arm5vte component; it
was a leftover from debugging.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-08-03 11:19:59 +02:00
Gilles Peskine c111e24292 Improve the detection of keep-going commands
Have simpler patterns related to 'test' (the central objective being to keep
going if 'make test' or 'tests/...' fails, but not if 'make tests' fails).

Add 'cd' as a can't-keep-going command.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-02 23:29:53 +02:00
Gilles Peskine 88a7c2b32e Improve --error-test reporting
Count invocations from 1 to n instead of n to 1.

Explain how changing the loop variable would cause an error if the function
was not executed in a subshell.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-02 23:28:00 +02:00
Gilles Peskine ec135544c8 Clarify some comments
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-02 23:27:42 +02:00
Hanno Becker 6c53ecc01d all.sh: Run basic TLS 1.3 with and without record padding
Signed-off-by: Hanno Becker <hanno.becker@arm.com>
2021-08-02 04:54:03 +01:00
Gilles Peskine 568f53a9d8 Don't unconditionally restore **/Makefile
all.sh restores **/Makefile from git in case the version in the worktree was
from doing a cmake in-tree build. Instead of doing this unconditionally, do
it only if the toplevel Makefile seems to have been automatically
generated (by cmake or otherwise, e.g. by mbedtls-prepare-build). This way
all.sh no longer silently wipes changes made to Makefile but not committed yet.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:39 +02:00
Gilles Peskine f83eb82a4d Don't restore *config.h before backing it up
Back up the config files at the beginning of all.sh, rather than before each
component. In particular, create the backup before running cleanup for the
first time. This fixes #3139 (all.sh using a config.h.bak from a previous
job), and makes all.sh more robust against accidentally using a modified
config.h midway through because a component messed with the backup.

Use a different extension (*.all.bak rather than *.bak) for the backups.
This is necessary to ensure that auxiliary scripts such as depends*.pl that
make their own backup don't remove all.sh's backup, which the code from this
commit does not support.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:31 +02:00
Gilles Peskine 03ab544832 Generate cpp_cummy_build.cpp dynamically
Generate programs/test/cpp_dummy_build.cpp dynamically instead of
maintaining it manually. This removes the need to update it when the list of
headers changes.

Include all the headers unconditionally except for the ones that cannot be
included directly.

Support this dynamic generation both with make and with cmake.

Adapt all.sh accordingly. Remove the redundant C build from
component_build_default_make_gcc_and_cxx (it was also done in
component_test_default_out_of_box), leaving a component_test_make_cxx. Also
run the C++ program, because why not. Do this in the full configuration
which may catch a bit more problems in headers.

Fixes #2570 for good.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:31 +02:00
Gilles Peskine 7238503642 Heed --quiet when running make generated_files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:31 +02:00
Gilles Peskine 88a07457c7 Remove barely-used redirect functions
redirect_out was no longer used and redirect_err was only used to
quiet dd. Change the dd invocation to only print diagnostics on
error (on platforms where this is possible).

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:31 +02:00
Gilles Peskine aca0b32132 Keep going after a shell "[" a.k.a. "test" fails
This is necessary to actually keep going and finish the
component-specific cleanup in component_test_cmake_out_of_source if
ssl-opt.err is non-empty.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:31 +02:00
Gilles Peskine c2e22ee271 Remove code that is useless now that components run in a subshell
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-08 18:49:19 +02:00
Gilles Peskine b80f0d20ea Complain if an unsupported component is explicitly requested
In all.sh, when an explicit list of components is specified, error out
if one of the components is not known or not supported. Patterns that
happen to match zero components are still effectively ignored.

Fix #2783

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-08 18:49:11 +02:00
Gilles Peskine a681c59d34 Better not function
In the `not` function, in keep-going mode, arrange to report the
failing command (rather than `"$@"`).

Note that the `!` keyword should not be used, because failures with
`!` are not reported properly.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-08 18:48:26 +02:00
Gilles Peskine fec30cbe8c Fix double reporting when the last command of a function fails
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-08 18:44:46 +02:00
Gilles Peskine 1f0cdaf3af Stop dispatching through obsolete functions
Remove the obsolete functions record_status and if_build_succeeded.
They didn't affect error detection, but they made error reporting
worse since $BASH_COMMAND would be the unexpanded "$@".

Keep the function definitions for the sake of pull requests using them
that may still be in flight.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-08 18:44:36 +02:00
Gilles Peskine f7e956c85c component_test_cmake_out_of_source: simplify and fix error handling
Remove ssl-opt.err even if it's empty.

Call cat unconditionally: it'll have no visible effect if the file is
empty.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-08 18:07:44 +02:00
Gilles Peskine 3664780f98 Detect errors on the left-hand side of a pipeline
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-08 18:07:36 +02:00
Gilles Peskine ce266c48bb Run each component in a subshell and handle errors more robustly
This commit completely rewrites keep-going mode. Instead of relying
solely on "set -e", which has some subtle limitations (such as being
off anywhere inside a conditional), use an ERR trap to record errors.

Run each component in a subshell. This way a component can set
environment variables, change the current directory, etc., without
affecting other components.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-08 18:07:20 +02:00
Gilles Peskine 5d99682a8c Add --error-test option to test error detection and reporting
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-08 17:35:19 +02:00
Gilles Peskine 62cf2e8e9f Switch all.sh to bash
This will let us use bash features that are not found in some other sh
implementations, such as DEBUG and ERR traps, "set -o pipefail", etc.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-08 17:35:19 +02:00
Bence Szépkúti 414d6bd424 Fix pre-existing typo in comment
Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2021-06-28 14:11:11 +01:00
Bence Szépkúti bb0cfeb2d4 Rename config.h to mbedtls_config.h
This commit was generated using the following script:

# ========================
#!/bin/sh
git ls-files | grep -v '^ChangeLog' | xargs sed -b -E -i '
s/((check|crypto|full|mbedtls|query)_config)\.h/\1\nh/g
s/config\.h/mbedtls_config.h/g
y/\n/./
'
mv include/mbedtls/config.h include/mbedtls/mbedtls_config.h
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2021-06-28 09:28:33 +01:00
Dave Rodgman 10bda58b49
Merge pull request #4259 from CJKay/cmake-config
Add CMake package config file
2021-06-25 20:32:13 +01:00
Ronald Cron 3698fa1043
Merge pull request #4673 from gilles-peskine-arm/psa_crypto_spm-from_platform_h
Fix and test the MBEDTLS_PSA_CRYPTO_SPM build
2021-06-25 09:01:08 +02:00
Manuel Pégourié-Gonnard 32750ef5c2
Merge pull request #4685 from mpg/improve-all-sh-robustness
all.sh: Clean up old files before generating them
2021-06-22 11:14:49 +02:00
Manuel Pégourié-Gonnard a805d57261
Merge pull request #4588 from TRodziewicz/remove_MD2_MD4_RC4_Blowfish_and_XTEA
Remove MD2, MD4, RC4, Blowfish and XTEA
2021-06-22 09:27:41 +02:00
Manuel Pégourié-Gonnard 87db8a2676 Clean up old files before generating them
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-06-18 13:30:14 +02:00
Gilles Peskine a354867399 In the SPM test build, fail if a symbol wasn't renamed
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-06-17 11:40:51 +02:00
Gilles Peskine d1dcfd53aa Do a test build with MBEDTLS_PSA_CRYPTO_SPM
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-06-17 11:40:51 +02:00
Manuel Pégourié-Gonnard d51aaad4c9 Remove config option MBEDTLS_ECP_NO_INTERNAL_RNG
It was used to remove the code used when mbedtls_ecp_mul() received a
NULL RNG parameter. This code is no longer relevant (as the RNG may no
longer be NULL) and will be unconditionally removed in the next commit.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-06-17 09:38:38 +02:00
TRodziewicz 10e8cf5fef Remove MD2, MD4, RC4, Blowfish and XTEA
Signed-off-by: TRodziewicz <tomasz.rodziewicz@mobica.com>
2021-06-16 10:34:25 +02:00
Gilles Peskine 1628a9c140 MBEDTLS_DEBUG_C is compatible with every whole-module ALT except DHM
It would be possible to make SSL debugging compatible with MBEDTLS_DHM_ALT,
but too much low-priority work right now, so don't require it.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-06-15 00:14:28 +02:00
Gilles Peskine 5c3f18d37c MBEDTLS_PK_PARSE_EC_EXTENDED is incompatible with MBEDTLS_ECP_ALT
... unless the alt implementation defines a group structure that's mostly
compatible with the built-in one and supports partially filled group
structures in the same way.

It would be possible to rewrite the SpecifiedECDomain parsing code to avoid
requiring support for partially filled group structures, but that's too
complicated to do now.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-06-15 00:13:02 +02:00
Gilles Peskine cc73cc55e8 Test the build with whole-module alternative implementations
Use headers defining dummy context types.

The test does not pass yet. I plan to fix this in subsequent commits.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-06-15 00:12:18 +02:00