They aren't used anywhere.
Keep the command line options of all.sh to avoid breaking any wrapper
scripts that people might have.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
None of the tests actually need GNUTLS_LEGACY (3.3.8): GNUTLS (3.4.10)
works.
None of the tests actually need OPENSSL_LEGACY (1.0.1j): OPENSSL (1.0.2g)
works.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use CMake to build the library out-of-source (rather than make)
in tests/scripts/test_psa_compliance.py and add a script argument for
the out-of-source build directory.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
Introduce changes needed to run all.sh in the psa-crypto repo. Where
behaviour must differ, detect that we are in the psa-crypto repo by
checking for the 'core' directory.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
* Don't break string literals in the allow list definition
* Comment each test that belongs to the allow list is there.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
* Turn the warnings produced when finding non-executed tests that
are not in the allow list into errors.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
Some basic test coverage for now:
* Nominal operation.
* Larger output buffer.
* Clone an operation and use it after the original operation stops.
Generate test data automatically. For the time being, only do that for
hashes that Python supports natively. Supporting all algorithms is future
work.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Since most of the code in "ECC+FFDH without BN" scenario was shared
with the "ECC without BN" one, I tried to reuse part of the code in
order to avoid duplications.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
Introduce the --require-full-coverage in analyze_outcomes.py so that
when analyze_outcomes.py --require-full-coverage is called, those
tests that are not executed and are not in the allowed list issue an
error instead of a warning.
Note that it is useful to run analyze_outcomes.py on incomplete test
results, so this error mode needs to remain optional in the long
term.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The allow list explicits which test cases are allowed to not be
executed when testing. This may be, for example, because a feature
is yet to be developed but the test for that feature is already in
our code base.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
component_test_crypto_full_no_cipher doesn't bring any extra value given the
existence of component_test_full_no_cipher.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
- `grep '^flags' /proc/cpuino` does not work in my local machine inside script.
- `make test programs/sleftest ` causes `strings | grep ` fail.
For timebeing, I did not figure out the root cause.
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
There is a precedent for having bigger and less mature options in
mbedtls_config.h (MBEDTLS_USE_PSA_CRYPTO) for an extended period.
Having this option in mbedtls_config.h is simpler and more robust.
Signed-off-by: Janos Follath <janos.follath@arm.com>
This previous test is correct, as it is using make. Switch to using
config.py for robustness and consistency.
Signed-off-by: Janos Follath <janos.follath@arm.com>
These tests weren't working, because they use CMake and can't pass
options with CFLAGS directly. This could be mitigated by adding a CMake
option, but using config.py is less invasive and it is what we normally
use for setting build options anyway.
Signed-off-by: Janos Follath <janos.follath@arm.com>
The latest and earliest clang/GCC compilers are now used through
variables instead of symlinks and also the all.sh script is updated
to support options for overriding the default values.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
The component functions in all.sh will be listed using
compgen instead of sed so this check is not needed.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
Explain how PSA_CRYPTO_DRIVER_TEST_ALL works and why we have it. Note that
it is incompatible with MBEDTLS_PSA_CRYPTO_CONFIG. As a consequence, disable
that option in component_test_psa_crypto_drivers.
MBEDTLS_PSA_CRYPTO_BUILTIN_KEYS is in the full config, so there's no need to
add it explicitly.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Running clang-16 on mbedtls reports warnings of type "-Wstrict-prototypes".
This patch fixes these warnings by adding void to functions with no
arguments. The generate_test_code.py is modified to insert void into test
functions with no arguments in *.function files.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
The Ubuntu 16.04 and 22.04 docker images have been updated with
earliest and latest versions of gcc and clang respectively.
This patch adds the necessary component and support functions
required for the CI to run these compilers.
For FreeBSD we invoke the function by name so a condition is added
to disable the existing test_clang_opt function for linux.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
- RSA was missing the MBEDTLS_ prefix.
- DH needs the same temporary fix (prefix + suffix) for now.
- hack_dependencies_not_implemented() needs to ignore MBEDTLS_PSA_WANT
dependencies.
While at it, make the code currently used for ECC more generic, so that
it's ready to be used for RSA and DH in the near future.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
ShebangIssueTracker implements the rule that scripts must be executable if
and only if they have a shebang line. By removing PermissionIssueTracker, we
now allow files with any extension to be executable (provided they have a
shebang line), and allow *.sh and *.pl to be non-executable modules if they
don't have a shebang line (as was already the case for *.py).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Run the sample program demo scripts in builds with a configuration
that is at least as complete as the default configuration.
Do not run sample programs in all configurations since they are
expected to fail if a required feature is missing.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
run_demos.py is the frontend to a framework for smoke-testing the
sample programs. It runs scripts called programs/*/*_demo.sh
("demo scripts") and check that they succeed. A typical demo script
runs one sample program or a combination of sample programs to
demonstrate their usage.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When MBEDTLS_USE_PSA_CRYPTO is disabled on a base of full, mention it.
Now that full implies MBEDTLS_PSA_CRYPTO_CONFIG, don't mention it, and don't
set it explicitly.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Having a wrapper made it harder to use: incompatible with setting
MBEDTLS_CONFIG_FILE, harder to combine with other settings. It was also
surprising since it was the only test config that was structured in that
way.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The original goal (https://github.com/Mbed-TLS/mbedtls/pull/5072) was to run
a test with ChaChaPoly disabled in PSA. It was actually implemented with GCM
also partially disabled (legacy GCM enabled but PSA GCM disabled), which
distracted from the objective. It's actually useful to test both with and
without GCM, so test both. Don't test inconsistencies between legacy and PSA
support because that's not a common case and not one we have particular
reasons to test.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
There were two copies of component_test_psa_crypto_config_accel_pake,
identical except for two typos. Keep the copy without the typos.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Note: the DERIVE option is intentionally skipped from the acceleration
list because this feature is still not supported.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
For now RSA key pairs' dependencies are kept to LEGACY, but this
is going to be updated in #7772.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
This includes:
- removal of temporary patches recently introdued in PR 7641
- proper replacing of legacy KEY_PAIR symbols with either
KEY_PAIR_IMPORT or KEY_PAIR_GENERATE depending on the test
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
- this component should work also for RSA and DH keys, not only for
EC
- of course the acceleated counterpart is hanlded properly
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
- Full tests generated by script only for ffdhe2048 group
- Single G->m and m->G exchange test for each other group
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com>
This includes also:
- auto enabling ECP_LIGHT when MBEDTLS_PK_PARSE_EC_COMPRESSED is
defined
- replacing ECP_LIGHT guards with PK_PARSE_EC_COMPRESSED in pkparse
- disabling PK_PARSE_EC_COMPRESSED in tests with accelarated EC curves
(it get disabled also in the reference components because we want
to achieve test parity)
- remove skipped checks in analyze_outcomes.py
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
ECP_LIGHT was never set as public symbol so it should not be
enabled/disabled using the config.py script.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
This is a temporary fix for not excluding tests which depend on
MBEDTLS_PSA_WANT_KEY_TYPE_[RSA/ECC]_KEY_PAIR_LEGACY. This fix
can be reverted as soon as those _LEGACY symbols will be removed
from the code.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
This is a temporary fix for replacing
PSA_WANT_KEY_TYPE_[RSA/ECC]_KEY_PAIR
with the temporary symbols
MBEDTLS_PSA_WANT_KEY_TYPE_[RSA/ECC]_KEY_PAIR_LEGACY.
Once new PSA_WANT_KEY_TYPE_[RSA/ECC]_KEY_PAIR_yyy will be used
both in library's code and tests, then this fix will be removed.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
Add FFDH support to the test case generator script: generate_tls13_compat_tests.py.
Add dependency for openssl as FFDH is supported from version 3.0.
Signed-off-by: Przemek Stekiel <przemyslaw.stekiel@mobica.com>
We no longer support TLS 1.0, TLS 1.1 or CBC record splitting since 3.0,
but those where still referenced here.
While at it, group things in a more logical way in this component.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Several components needed to remove it, with the same comment every
time. It's probably just chance that other components happened to work
despite it being enabled.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Compared to the previous scheme, this avoid the problem of having to
warn about adjusting PSA_WANT in the wrong place.
Also, it allows enabling MBEDTLS_PSA_CRYPTO_CONFIG in adjust_config
rather than having to repeat it in every single component.
It also plays more nicely with components that have an associated
reference component and use a common config function. (Some of them were
already using the new order.)
Finally, "configure, build, run the tests" seems more natural than
"configure, build, configure, build, test" (and, coming back to the
initial point, it avoid questions about what to configure when).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
ARMC5_BIN_DIR and ARMC6_BIN_DIR were set in pre_parse_command_line() and used
by support_build_armcc() which is called by pre_initialize_variables() to
determines SUPPORTED_COMPONENTS.
As pre_initialize_variables() is called before pre_parse_command_line(),
support_build_armcc() failed to use the directories set on the command line.
However, we can't call pre_parse_command_line() before pre_initialize_variables()
since the former needs SUPPORTED_COMPONENTS!
Fix the circular dependency by parsing the command line twice, with the first
pass only to get these directories.
Signed-off-by: Tom Cosgrove <tom.cosgrove@arm.com>
The exception in analyze_outcomes.py follows previous commit in which
a test in test_suite_pkparse was set with the ECP_C guard for a different
parsing of the private key between the legacy and PSA implementations.
The wrong guard in test_suite_ecp.function instead was erroneously
added in a past commit and it was setting a non-existing symbol of
mbedTLS so those tests were basically never executed.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
We maintain a dict with unique AudiData objects
(AuditData with unique underlying X.509 objects).
We don't need merge_auditdata anymore.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
We should print all the information for each objects
found every line. This makes it easy to analyze the
output.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
The identifier is calculated SHA1 hex string from
the DER encoding of each X.509 objects. It's useful
for finding out the identical X.509 objects.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
check_cryptography_version didn't provide helpful
message with Python < 3.6, because re.Match object
is not subscriptable.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
Previously, if a file had multiple PEM objects, only
the first one would be parsed. This commit add the
support so that we could parse all the PEM objects
in the file.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
This commit adds support to auto-enable
MBEDTLS_CTR_DRBG_USE_128_BIT_KEY if
MBEDTLS_AES_ONLY_128_BIT_KEY_LENGTH enabled.
Furthermore, the corresponding check is removed in check_config.h.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
This commit introduces a new function to generate extra dependencies
based on REGEX search pattern and predefined dependency list.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Until now, we were never enabling this option in any test.
MBEDTLS_PSA_INJECT_ENTROPY requires MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES, so
it cannot be enabled in the full config and it gets its own component.
Test with MBEDTLS_USE_PSA_CRYPTO enabled, since MBEDTLS_PSA_INJECT_ENTROPY
is a very PSA feature (which can break non-PSA applications), and Mbed
OS (for whch MBEDTLS_PSA_INJECT_ENTROPY was designed) enables
MBEDTLS_USE_PSA_CRYPTO when it enables MBEDTLS_PSA_INJECT_ENTROPY.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Now that the C code supports the full range of intmax_t, allow any size of
signed integer type in the .data file parser.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Change the type of signed integer arguments from int32_t to intmax_t.
This allows the C code to work with test function arguments with a range
larger than int32_t. A subsequent commit will change the .datax generator
to support larger types.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use normalization the equality comparisons instead of loose regular
expressions to determine the type of an argument of a test function.
Now declarations are parsed in a stricter way: there can't be ignored junk
at the beginning or at the end. For example, `long long unsigned int x`
was accepted as a test function argument (but not `long long unsigned x`),
although this was misleading since the value was truncated to the range of
int. Now only recognized types are accepted.
The new code is slightly looser in that it accepts `char const*` as well as
`const char*`.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The test framework stores size_t and int32_t values in the parameter store
by converting them all to int. This is ok in practice, since we assume int
covers int32_t and we don't have test data larger than 2GB. But it's
confusing and error-prone. So make the parameter store a union, which allows
size_t values not to be potentially truncated and makes the code a little
clearer.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In the .datax parser, since we're calling strtol() anyway, rely on it for
verification. This makes the .datax parser very slightly more
liberal (leading spaces and '+' are now accepted), and changes the
interpretation of numbers with leading zeros to octal.
Before, an argument like :0123: was parsed as decimal, but an argument like
:0123+1: was parsed as a C expression and hence the leading zero marked an
octal representation. Now, a leading zero is always interpreted according to
C syntax, namely indicating octal. There are no nonzero integer constants
with a leading zero in a .data file, so this does not affect existing test
cases.
In the .datax generator, allow negative arguments to be 'int' (before, they
were systematically treated as 'exp' even though they didn't need to be).
In the .datax parser, validate the range of integer constants. They have to
fit in int32_t. In the .datax generator, use 'exp' instead of 'int' for
integer constants that are out of range.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
No intended behavior change. This commit is mainly to satisfy pylint, which
complains that gen_from_test_data now has too many variables. But it's a
good thing anyway to make the function a little more readable.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The script requires cryptography >= 35.0.0, we
need to check the version and provide meaningful
error message when the package version was too
old.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
The commit adds '--data-files' and '--suite-data-files'
options so that we could pass names for the two types
of files separately. Additionally, the commit improves
the documentation in the script.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
This commit is a collection of improving the documentation in the
script:
* Restore uppercase in the license header.
* Reword the script description.
* Reword the docstring of AuditData.fill_validity_duration
* Rename AuditData.filename to *.location
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
Make FILE as positional argument so that we can
pass multiple files to the script. This commit
also contains some help message improvements.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
Only some test cases are skipped for which ECP_C is mandatory,
but the other ones are included.
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
This is to make CI happy. The script requires cryptography
>= 35.0.0, which is only available for Python >= 3.6. But
both ubuntu-16.04 and Travis CI are using Python 3.5.x.
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
