Hi,
I coded a high performance sha256 algorithm for ppc64le:
https://git.lysator.liu.se/gut/nettle/commit/a8facb03a69787a93c91b426f32a0b…
Tests were performed with different files and comparing it against the original C implementation by using the Ubuntu's 16.10 libnettle.so.6 by using the following code:
https://gist.github.com/gut/9622d4535a9e3f9ea3b0ded2762d4b28
The results I got by using an ubuntu-16.10 iso as an input (around 700mb):
$ time ./sha256-test ~/ubuntu-16.10-server-ppc64el.iso
d14bdb413ea6cdc8d9354fcbc37a834b7de0c23f992deb0c6764d0fd5d65408e
real 0m14.607s
user 0m13.988s
sys 0m0.616s
$ export LD_LIBRARY_PATH=~/nettle # link with user and not system lib
$ time ./sha256-test ~/ubuntu-16.10-server-ppc64el.iso
d14bdb413ea6cdc8d9354fcbc37a834b7de0c23f992deb0c6764d0fd5d65408e
real 0m2.242s
user 0m2.052s
sys 0m0.188s
I also plan on doing the sha512 if I'm heading to the right direction.
Regards,
Gustavo Serra Scalet
> -----Original Message-----
> From: Nikos Mavrogiannopoulos [mailto:n.mavrogiannopoulos@gmail.com]
> Sent: terça-feira, 28 de fevereiro de 2017 06:07
> To: Niels Möller <nisse(a)lysator.liu.se>
> Cc: Gustavo Serra Scalet <gustavo.scalet(a)eldorado.org.br>; nettle-
> bugs(a)lists.lysator.liu.se
> Subject: Re: Is the gitlab being used or not?
>
> On Tue, Feb 28, 2017 at 7:48 AM, Niels Möller <nisse(a)lysator.liu.se>
> wrote:
> > Gustavo Serra Scalet <gustavo.scalet(a)eldorado.org.br> writes:
> >
> >> I noticed there is no merging/update activities on the gitlab
> website:
> >> https://git.lysator.liu.se/nettle/nettle/merge_requests?scope=all&sta
> >> te=all
> >>
> >> I wonder if it's used.
> >
> > It's used mainly for the plain git hosting. I rarely use the more
> > advance features, and almost never access the web interface.
>
> Would it be possible to disable then the merge request option in the
> gitlab interface? There are already 5 patches there.
>
> Most likely you need to go to -> Edit project and disable the issue
> tracker and the merge commits for everyone (at least that's the process
> in the new gitlab).
>
> >> The reason for asking is that I'm willing to integrate a
> >> ppc64le-specific implementation using SIMD registers for the SHA-2
> >> algorithm so I would create an Issue to discuss it and then
> >> submitting the code as a Merge Request. But only if that's the way
> >> for upstreaming code on nettle. If not, please advise if this list
> >> can be used.
> >
> > Discussion should take place on this list. You can mail patches here,
> > or create a merge request on gitlab. But in the latter case, please
> > send a note here too, otherwise it might be overlooked.
>
> I attach a patch to document that process. I use CONTRIBUTION.md show it
> shows prominently on the gitlab interfaces.
>
> regards,
> Nikos
Hi,
I noticed there is no merging/update activities on the gitlab website:
https://git.lysator.liu.se/nettle/nettle/merge_requests?scope=all&state=all
I wonder if it's used.
The reason for asking is that I'm willing to integrate a ppc64le-specific implementation using SIMD registers for the SHA-2 algorithm so I would create an Issue to discuss it and then submitting the code as a Merge Request. But only if that's the way for upstreaming code on nettle. If not, please advise if this list can be used.
Sorry for using the "nettle-bugs" mailing list, which is not a discussion mailing list.
Cheers,
Gustavo Serra Scalet
On Sat, Mar 4, 2017 at 2:31 PM, Aapo Talvensaari
<aapo.talvensaari(a)gmail.com> wrote:
> On Sat, Mar 4, 2017 at 12:18 PM, Gabriel Ivașcu <ivascu.gabriel59(a)gmail.com>
> wrote:
>>
>> Hi,
>>
>> Can I get some instructions on how to use an IV (initialization
>> vector) when doing an AES 256 encryption/decryption?
>>
>> The AES documentation [0] provides no information on how to pass the
>> IV to the AES functions,
>
>
> Please check here:
> https://github.com/bungle/lua-resty-nettle/blob/master/lib/resty/nettle/aes…
>
Thanks for your response, Aapo!
I learned eventually that you have to explicitly use the CBC mode [0]
if you want to pass an IV, since the AES functions [1] use the ECB
mode as default.
Also, I found this useful example [2] of how use the CBC mode with
Twofish, which I adapted to use AES instead.
[0] https://www.lysator.liu.se/~nisse/nettle/nettle.html#CBC
[1] https://www.lysator.liu.se/~nisse/nettle/nettle.html#AES
[2] https://stackoverflow.com/questions/33003118/nettle-twofish-cbc
Regards,
Gabriel
Hi,
Can I get some instructions on how to use an IV (initialization
vector) when doing an AES 256 encryption/decryption?
The AES documentation [0] provides no information on how to pass the
IV to the AES functions.
[0] https://www.lysator.liu.se/~nisse/nettle/nettle.html#AES
Thank you,
Gabriel
Hi,
One of the items that have been on my todo-list after discussing with
application writers of multi-threaded applications (mainly servers),
is addressing the issue of synchronization for the random generator.
Currently gnutls provides a "central" random generator based on yarrow
(for keys) and salsa20 (for nonces) primitives, and it is thread safe
by utilizing mutexes over it. An application that has more than
100-200 threads is most likely to spend more time in synchronization
rather than the random generator itself. A solution to that would be
to provide a thread-local random generator which will work lock-free,
at the cost of additional memory per-thread -around 600-700 bytes for
the current generator-.
I have an experimental patch set, implementing this idea at:
https://gitlab.com/gnutls/gnutls/merge_requests/259
On the patch above, the additional cost per thread will only be for
threads actually utilizing gnutls, and in particular the random
generator, as the required memory will be allocated after the first
call to gnutls_rnd() by the thread.
Are there any objections on such an enhancement to gnutls, or
suggestions on how such a lock-free random generator could be improved
(in terms of memory utilization for example)?
regards,
Nikos
Hi,
There is a sha1 collision detection library at:
https://github.com/cr-marcstevens/sha1collisiondetection
Given that algorithms like sha1 are notoriously hard to phase out,
does it make sense for nettle to add support for collision detection,
in effect giving applications the option to switch to a mode where a
known to be insecure hash is enabled, but with such collision detection
mitigations in place.
Does something like the above library make sense to be added to nettle?
regards,
Nikos
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Aloha!
Ran into the Google OSS-Fuzz project. The project provides the Google
compute power for oss projects to run fuzzing on the code. Judging by
the list of projects libraries such as GnuTLS, OpenSSL, LibSSH and Curl
are being tested. Would it be worthwile to try and get Nettle accepted
as a project? Has anybody looked at OSS-Fuzz for Nettle?
https://github.com/google/oss-fuzzhttps://github.com/google/oss-fuzz/tree/master/projects
Another imho interesting project Google project is Wycheproof. The
project tests crypto libraries against known attacks/issues such as
invalid curve attacks, biased nonces in digital signature schemes etc.
Right now there are 80 tests. The code is in Java and I don't know how
easy/hard it would be to connect Nettle to the test system. But at least
one could look at the tests and implement the same tests for Nettle.
https://github.com/google/wycheproofhttps://github.com/google/wycheproof/tree/master/java/com/google/security/w…
Has anybody looked at Wycheproof for testing Nettle?
- --
Med vänlig hälsning, Yours
Joachim Strömbergson - Alltid i harmonisk svängning.
========================================================================
Joachim Strömbergson Secworks AB joachim(a)secworks.se
========================================================================
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQIcBAEBCAAGBQJYf23WAAoJEF3cfFQkIuyNKVgP/1R08TRJCYlMR2LsN3HxEJLj
P6ZdksGL1A6jWHlSV7iN5UNJKWGXmF3mJ3mo4AgBZD1Xe6t1S6cpn4RkyFgc51xa
+JDJgEeMhxGZptfCSjwktnoBo7T6WYZzPlw2SghD9ur6D2wiJ71K/P6LpD+3omLj
BrUbaihPvRKMTPlkZi33cgjj9L1ni9LPf1I7Ir1GibtrT2NNZKZ3l2pZ5dHt7sr5
5ZtXlQZPcIlUJkpA0EUUgV5vnXMr4MxR4NFphY3xRh562ffILV0HPO+bjjPJCDo/
qbLdq0jeDXxHoIjrhUgCKWZ+03zvE+Qi+3iBiGziOS+NC92nXOnni9ehfmHrvEe5
HDBZmtO8otkU0WuUt88ovjoOKtjkvKV05gfNhwDDLSGL2NlYllAsF/13h5Ewsi2f
huHdo9QPO9gUDwMDfjXt58zGmlGipju3ybSj1K9Ly/9uD1ep/yLXA/L6w2iFZ7ga
XPBf0StjDLwF7yMgmhXhhhKAbV0+SgmdghG1z4D5jEVnZRhMpKN4KUs9seS8wznL
XyqhWxC/swiKP1BEMbpIW9XZ3hJaMb8bKqObv5BWXscrlV7aStv9o5CFmgSeaPBr
AJHXk+w6Q1dXbaVp+Tb4uhzNBVyHcLkiitQYITW8mg6SwoFclra3oh2Es2XZU1c9
h6K29UbA7UaGyum5ZDED
=yz1M
-----END PGP SIGNATURE-----
The idea of the interface
/* null-terminated list of digests implemented by this version of nettle */
extern const struct nettle_hash * const nettle_hashes[];
was that the size of the array isn't part of the ABI; a new Nettle version
should be able to extend it with more entries, without breaking the ABI.
However, I've recently learnt that it does break, in a subtle way. (See
https://bugs.gentoo.org/show_bug.cgi?id=601512 and
http://trofi.github.io/posts/195-dynamic-linking-ABI-is-hard.html).
The problematic case is a traditional non-PIC executable linking with a
nettle shared library; in this post, I care exclusively about this case.
All symbol references in the executable are resolved at link time, and
at load time it is mapped at a fixed address (traditionally 0, I don't
know if current systems skip the first page), without any relocations.
Now, if the executable links with libnettle.so, and contains a reference
to the symbol nettle_hashes, the address where nettle.so is going to be
mapped isn't known at link time? So how can the executable refer to it
without load-time relocation of its references? The linker solves that
problem using the curious relocation type R_X86_64_COPY.
What the linker does is that it allocates space for a copy of the data
in the BSS segment of the executable, and resolve all references to
point to that copy. At the time libnettle.so is loaded, the list in the
library is copied (after relocating it, but that's a minor complication
in this context) to the space in the BSS. I imagine the dynamic linker
also adjusts the pointers in libnettle.so's GOT table to refer to the
copy rather than the original.
Now the problem is that the allocation in the BSS segment, as well as
the copying operation, are based on the size of the data object as
recorded in the version of libnettle.so available at the time the
executable was linked.
If the array size is larger in the version of libnettle.so actally
loaded, the copy operation is truncates it, which is particularly bad
when it's NULL-terminated. So the array size, which was intended to not
be part of the ABI, creeps into the ABI.
So what to do about this? We have to break the ABI, but I'd prefer if we
keep the API unchanged. Some alternatives:
1. Define nettle_hashesp as a constant pointer to the current
nettle_hashes list, and
#define nettle_hashes (*nettle_hashesp)
Then nettle_hashesp will still get a R_X86_64_COPY relocation, but now
the size is always a single pointer, regardless of the array size. At
load time, it will be set to point directly to the list in the
data segment of the loaded libnettle.so.
2. Define a function get_nettle_hashes returning a pointer to the list,
and
#define nettle_hashes (get_nettle_hashes())
In this case, the indirection is via a PLT entry in the executable.
3. Define the array with a size explicitly part of the ABI,
extern const struct nettle_hash * const nettle_hashes[17];
Add a some reasonable number of reserved NULL entries at the end, and
make an ABI break whenever we run out of reserved places and have to
increase the size.
We also have other public data, e.g., individual nettle_hash structs,
like
extern const struct nettle_hash nettle_sha256;
These will also get a R_X86_64_COPY relocation if referenced (and all
internal references within libnettle.so will be relocated to the copy in
the executable's BSS, I guess). But that's less of a problem, since the
size and layout is already part of the ABI.
More problematic are the objects declared in ecc-curve.h; the size and
layout of struct ecc_curve was intended to be an implementation detail,
not part of the ABI, but isn't, when R_X86_64_COPY is involved.
Advice appreciated. I'd also like to hear if anyone on the list knows
how these things work with windows dlls. I've read sometime that
exporting data (in contrast to functions) from a dll is extra tricky,
but I don't remember any details. If we make any changes to fix the
exported-data issues with ELF, it would be good if we could ensure that
we solve any related problems for libnettle.dll too.
I understand why R_X86_64_COPY is needed, but it works in a way that was
pretty counter-intutive to me. The effect is, more or less, that the
library's data is *statically* linked into the executable. And then
initialized at load time based on the contents of the loaded library.
And we then get a mix of statically linked and dynamically linked parts
which might originate in different versions of the library.
It would be prettier if we could force the executable to always access
library data via GOT (like it works for PIC code), and never use
R_X86_64_COPY. But I guess that has to be known at code generation time,
and likely too late to fixup at link time, which is when we know which
external data objects are defined by some shared library, and which
aren't. But if anyone knows how to fix the issue in this way, I'd be
delighted.
Regards,
/Niels
--
Niels Möller. PGP-encrypted email is preferred. Keyid 368C6677.
Internet email is subject to wholesale government surveillance.