commit 7459e1d25ffefa2b1be799477fcc1f6c62f6cec7 upstream.
Driver does not properly handle the case when signals interrupt
wait_for_completion_interruptible():
-it does not check for return value
-completion structure is allocated on stack; in case a signal interrupts
the sleep, it will go out of scope, causing the worker thread
(caam_jr_dequeue) to fail when it accesses it
wait_for_completion_interruptible() is replaced with uninterruptable
wait_for_completion().
We choose to block all signals while waiting for I/O (device executing
the split key generation job descriptor) since the alternative - in
order to have a deterministic device state - would be to flush the job
ring (aborting *all* in-progress jobs).
Cc: <stable@vger.kernel.org>
Fixes: 045e36780f ("crypto: caam - ahash hmac support")
Fixes: 4c1ec1f930 ("crypto: caam - refactor key_gen, sg")
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Willy Tarreau <w@1wt.eu>
commit a0118c8b2be9297aed8e915c60b4013326b256d4 upstream.
Since 6de62f15b581 ("crypto: algif_hash - Require setkey before
accept(2)"), the AF_ALG interface requires userspace to provide a key
to any algorithm that has a setkey method. However, the non-HMAC
algorithms are not keyed, so setting a key is unnecessary.
Fix this by removing the setkey method from the non-keyed hash
algorithms.
Fixes: 6de62f15b581 ("crypto: algif_hash - Require setkey before accept(2)")
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Willy Tarreau <w@1wt.eu>
commit d128af17876d79b87edf048303f98b35f6a53dbc upstream.
The AEAD givenc descriptor relies on moving the IV through the
output FIFO and then back to the CTX2 for authentication. The
SEQ FIFO STORE could be scheduled before the data can be
read from OFIFO, especially since the SEQ FIFO LOAD needs
to wait for the SEQ FIFO LOAD SKIP to finish first. The
SKIP takes more time when the input is SG than when it's
a contiguous buffer. If the SEQ FIFO LOAD is not scheduled
before the STORE, the DECO will hang waiting for data
to be available in the OFIFO so it can be transferred to C2.
In order to overcome this, first force transfer of IV to C2
by starting the "cryptlen" transfer first and then starting to
store data from OFIFO to the output buffer.
Fixes: 1acebad3d8 ("crypto: caam - faster aead implementation")
Signed-off-by: Alex Porosanu <alexandru.porosanu@nxp.com>
Signed-off-by: Horia GeantÄ <horia.geanta@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Willy Tarreau <w@1wt.eu>
commit b310c178e6d897f82abb9da3af1cd7c02b09f592 upstream.
When doing pointer operation for accessing the HW S/G table,
a value representing number of entries (and not number of bytes)
must be used.
Fixes: 045e36780f ("crypto: caam - ahash hmac support")
Signed-off-by: Horia Geant? <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
commit 412c98c1bef65fe7589f1300e93735d96130307c upstream.
The hwrng output buffers (2) are cast inside of a a struct (caam_rng_ctx)
allocated in one DMA-tagged region. While the kernel's heap allocator
should place the overall struct on a cacheline aligned boundary, the 2
buffers contained within may not necessarily align. Consenquently, the ends
of unaligned buffers may not fully flush, and if so, stale data will be left
behind, resulting in small repeating patterns.
This fix aligns the buffers inside the struct.
Note that not all of the data inside caam_rng_ctx necessarily needs to be
DMA-tagged, only the buffers themselves require this. However, a fix would
incur the expense of error-handling bloat in the case of allocation failure.
Signed-off-by: Steve Cornelius <steve.cornelius@freescale.com>
Signed-off-by: Victoria Milhoan <vicki.milhoan@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
req->assoc is dma mapped BIDIRECTIONAL and unmapped TO_DEVICE.
Since it is read-only for the device, use TO_DEVICE both for mapping
and unmapping.
Cc: <stable@vger.kernel.org> # 3.9, 3.8
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Pull crypto update from Herbert Xu:
- XTS mode optimisation for twofish/cast6/camellia/aes on x86
- AVX2/x86_64 implementation for blowfish/twofish/serpent/camellia
- SSSE3/AVX/AVX2 optimisations for sha256/sha512
- Added driver for SAHARA2 crypto accelerator
- Fix for GMAC when used in non-IPsec secnarios
- Added generic CMAC implementation (including IPsec glue)
- IP update for crypto/atmel
- Support for more than one device in hwrng/timeriomem
- Added Broadcom BCM2835 RNG driver
- Misc fixes
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (59 commits)
crypto: caam - fix job ring cleanup code
crypto: camellia - add AVX2/AES-NI/x86_64 assembler implementation of camellia cipher
crypto: serpent - add AVX2/x86_64 assembler implementation of serpent cipher
crypto: twofish - add AVX2/x86_64 assembler implementation of twofish cipher
crypto: blowfish - add AVX2/x86_64 implementation of blowfish cipher
crypto: tcrypt - add async cipher speed tests for blowfish
crypto: testmgr - extend camellia test-vectors for camellia-aesni/avx2
crypto: aesni_intel - fix Kconfig problem with CRYPTO_GLUE_HELPER_X86
crypto: aesni_intel - add more optimized XTS mode for x86-64
crypto: x86/camellia-aesni-avx - add more optimized XTS code
crypto: cast6-avx: use new optimized XTS code
crypto: x86/twofish-avx - use optimized XTS code
crypto: x86 - add more optimized XTS-mode for serpent-avx
xfrm: add rfc4494 AES-CMAC-96 support
crypto: add CMAC support to CryptoAPI
crypto: testmgr - add empty test vectors for null ciphers
crypto: testmgr - add AES GMAC test vectors
crypto: gcm - fix rfc4543 to handle async crypto correctly
crypto: gcm - make GMAC work when dst and src are different
hwrng: timeriomem - added devicetree hooks
...
checkstack reports report_deco_status(), report_ccb_status() as
particularly excessive stack users. Move their lookup tables
off the stack and put them in .rodata.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
commit 2af8f4a "crypto: caam - coccicheck fixes" added error
return values yet neglected to change the type from unsigned.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This change is required for post SEC-5.0 devices which have RNG4.
Setting RDB in security configuration register allows CAAM to use the
"Random Data Buffer" to be filled by a single request. The Random Data
Buffer is large enough for ten packets to get their IVs from a single
request. If the Random Data Buffer is not enabled, then each IV causes a
separate request, and RNG4 hardware cannot keep up resulting in lower
IPSEC throughput if random IVs are used.
Signed-off-by: Vakul Garg <vakul@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This reverts commit 891104ed00.
Current IPsec ESN implementation for authencesn(cbc(aes), hmac(sha))
(separate encryption and integrity algorithms) does not conform
to RFC4303.
ICV is generated by hashing the sequence
SPI, SeqNum-High, SeqNum-Low, IV, Payload
instead of
SPI, SeqNum-Low, IV, Payload, SeqNum-High.
Cc: <stable@vger.kernel.org> # 3.8, 3.7
Reported-by: Chaoxing Lin <Chaoxing.Lin@ultra-3eti.com>
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Reviewed-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The Kconfig entry for CAAM's hash algorithm implementations has always
selected CRYPTO_AHASH. But there's no corresponding Kconfig symbol.
It seems it was intended to select CRYPTO_HASH, like other crypto
drivers do. That would apparently (indirectly) select CRYPTO_HASH2,
which would enable the ahash functionality this driver uses.
Signed-off-by: Paul Bolle <pebolle@tiscali.nl>
Reviewed-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
CONFIG_HOTPLUG is going away as an option. As a result, the __dev*
markings need to be removed.
This change removes the use of __devinit, __devexit_p, __devinitdata,
and __devexit from these drivers.
Based on patches originally written by Bill Pemberton, but redone by me
in order to handle some of the coding style issues better, by hand.
Cc: Bill Pemberton <wfp5p@virginia.edu>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Kent Yoder <key@linux.vnet.ibm.com>
Cc: Jamie Iles <jamie@jamieiles.com>
Cc: Kim Phillips <kim.phillips@freescale.com>
Cc: Shengzhou Liu <Shengzhou.Liu@freescale.com>
Cc: Alex Porosanu <alexandru.porosanu@freescale.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Pull crypto update from Herbert Xu:
- Optimised AES/SHA1 for ARM.
- IPsec ESN support in talitos and caam.
- x86_64/avx implementation of cast5/cast6.
- Add/use multi-algorithm registration helpers where possible.
- Added IBM Power7+ in-Nest support.
- Misc fixes.
Fix up trivial conflicts in crypto/Kconfig due to the sparc64 crypto
config options being added next to the new ARM ones.
[ Side note: cut-and-paste duplicate help texts make those conflicts
harder to read than necessary, thanks to git being smart about
minimizing conflicts and maximizing the common parts... ]
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (71 commits)
crypto: x86/glue_helper - fix storing of new IV in CBC encryption
crypto: cast5/avx - fix storing of new IV in CBC encryption
crypto: tcrypt - add missing tests for camellia and ghash
crypto: testmgr - make test_aead also test 'dst != src' code paths
crypto: testmgr - make test_skcipher also test 'dst != src' code paths
crypto: testmgr - add test vectors for CTR mode IV increasement
crypto: testmgr - add test vectors for partial ctr(cast5) and ctr(cast6)
crypto: testmgr - allow non-multi page and multi page skcipher tests from same test template
crypto: caam - increase TRNG clocks per sample
crypto, tcrypt: remove local_bh_disable/enable() around local_irq_disable/enable()
crypto: tegra-aes - fix error return code
crypto: crypto4xx - fix error return code
crypto: hifn_795x - fix error return code
crypto: ux500 - fix error return code
crypto: caam - fix error IDs for SEC v5.x RNG4
hwrng: mxc-rnga - Access data via structure
hwrng: mxc-rnga - Adapt clocks to new i.mx clock framework
crypto: caam - add IPsec ESN support
crypto: 842 - remove .cra_list initialization
Revert "[CRYPTO] cast6: inline bloat--"
...
we need to configure the TRNG to use more clocks per sample
to handle the two back-to-back 64KiB random descriptor requests
on higher frequency P5040s.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
According to SEC v5.0-v5.3 reference manuals.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Acked-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Support for ESNs (extended sequence numbers).
Tested with strongswan by connecting back-to-back P1010RDB with P2020RDB.
Signed-off-by: Horia Geanta <horia.geanta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
use true/false for bool, fix code alignment, and fix two allocs with
no test.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In 3.6-rc3, without this patch, the following error occurs with a modular build:
ERROR: "gen_split_key" [drivers/crypto/caam/caamhash.ko] undefined!
ERROR: "gen_split_key" [drivers/crypto/caam/caamalg.ko] undefined!
Signed-off-by: Ben Collins <ben.c@servergy.com>
Cc: Yuan Kang <Yuan.Kang@freescale.com>
Acked-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SHARE_WAIT, whilst more optimal for association-less crypto,
has the ability to start thrashing the CCB descriptor/key
caches, given high levels of traffic across multiple security
associations (and thus keys).
Switch to using the SERIAL sharing type, which prefers
the last used CCB for the SA. On a 2-DECO platform
such as the P3041, this can improve performance by
about 3.7%.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In some device trees of previous version, there were string "fsl,sec4.0".
To be backward compatible with device trees, we first check "fsl,sec-v4.0",
if it fails, then check for "fsl,sec4.0".
Signed-off-by: Shengzhou Liu <Shengzhou.Liu@freescale.com>
extended to include new hash and rng code, which was omitted from
the previous version of this patch during a rebase of the SDK
version.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds support for retrieving and printing of
SEC ERA information. It is useful for knowing beforehand
what features exist from the SEC point of view on a
certain SoC. Only era-s 1 to 4 are currently supported;
other eras will appear as unknown.
Signed-off-by: Alex Porosanu <alexandru.porosanu@freescale.com>
- rebased onto current cryptodev master
- made caam_eras static
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The caam job rings (input/output job ring) are allocated using
dma_map_single(). These job rings can be visualized as the ring
buffers in which the jobs are en-queued/de-queued. The s/w enqueues
the jobs in input job ring which h/w dequeues and after processing
it copies the jobs in output job ring. Software then de-queues the
job from output ring. Using dma_map/unmap_single() is not preferred
way to allocate memory for this type of requirements because this
adds un-necessary complexity.
Example, if bounce buffer (SWIOTLB) will get used then to make any
change visible in this memory to other processing unit requires
dmap_unmap_single() or dma_sync_single_for_cpu/device(). The
dma_unmap_single() can not be used as this will free the bounce
buffer, this will require changing the job rings on running system
and I seriously doubt that it will be not possible or very complex
to implement. Also using dma_sync_single_for_cpu/device() will also
add unnecessary complexity.
The simple and preferred way is using dma_alloc_coherent() for these
type of memory requirements.
This resolves the Linux boot crash issue when "swiotlb=force" is set
in bootargs on systems which have memory more than 4G.
Signed-off-by: Bharat Bhushan <bharat.bhushan@freescale.com>
Acked-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
there is no noticeable benefit for multiple cores to process one
job ring's output ring: in fact, we can benefit from cache effects
of having the back-half stay on the core that receives a particular
ring's interrupts, and further relax general contention and the
locking involved with reading outring_used, since tasklets run
atomically.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Memory barriers are implied by the i/o register write implementation
(at least on Power). So we can remove the redundant wmb() in
caam_jr_enqueue, and, in dequeue(), hoist the h/w done notification
write up to before we need to increment the head of the ring, and
save an smp_mb.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Code was needlessly checking the s/w job ring when there
would be nothing to process if the h/w's output completion
ring were empty anyway.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The enqueue lock isn't used in any interrupt context, and
the dequeue lock isn't used in the h/w interrupt context,
only in bh context.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
It has been observed that in zero-loss benchmarks, when a
slow traffic rate is being tested, the IRQ timer coalescing
parameter was set too high, and the ethernet controller
would start dropping packets because the job ring back half
wouldn't be executed in time before the ethernet controller
would fill its buffers, thereby significantly reducing the
zero-loss performance figures.
Empirical testing has shown that the best zero-loss performance
is achieved when IRQ coalescing is set to minimum values and/or
turned off, since apparently the job ring driver already implements
an adequately-performing general-purpose IRQ mitigation strategy
in software.
Whilst we could go with minimal count (2-8) and timing settings
(192-256), we prefer just turning h/w coalescing altogether off
to minimize setkey latency (due to split key generation), and
for consistent cross-SoC performance (the SEC vs. core clock
ratio changes).
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The SEC v4.x' RNGB h/w block self-initialized. RNG4, available
on SEC versions 5 and beyond, is based on a different standard
that requires manual initialization.
Also update any new errors From the SEC v5.2 reference manual:
The SEC v5.2's RNG4 unit reuses some error IDs, thus the addition
of rng_err_id_list over the CHA-independent err_id_list.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC v4.x were only 36-bit, SEC v5+ are 40-bit capable.
Also set a DMA mask for any job ring devices created.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
caam_read copies random bytes from two buffers into output.
caam rng can fill empty buffer 0xffff bytes at a time,
but the buffer sizes are rounded down to multiple of cacheline size.
Signed-off-by: Yuan Kang <Yuan.Kang@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
support chained scatterlists for aead, ablkcipher and ahash.
Signed-off-by: Yuan Kang <Yuan.Kang@freescale.com>
- fix dma unmap leak
- un-unlikely src == dst, due to experience with AF_ALG
Signed-off-by: Kudupudi Ugendreshwar <B38865@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
caam supports and registers unkeyed sha algorithms and md5.
Signed-off-by: Yuan Kang <Yuan.Kang@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
caam supports ahash hmac with sha algorithms and md5.
Signed-off-by: Yuan Kang <Yuan.Kang@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
- rename scatterlist and link_tbl functions
- link_tbl changed to sec4_sg
- sg_to_link_tbl_one changed to dma_to_sec4_sg_one,
since no scatterlist is use
Signed-off-by: Yuan Kang <Yuan.Kang@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
create separate files for split key generation and scatterlist functions.
Signed-off-by: Yuan Kang <Yuan.Kang@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
remove caam_jr_register and caam_jr_deregister
to allow sharing of job rings.
Signed-off-by: Yuan Kang <Yuan.Kang@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
functions for external storage of seq in/out lengths,
i.e., for 32-bit lengths.
These type-dependent functions automatically determine whether to
store the length internally (embedded in the command header word) or
externally (after the address pointer), based on size of the type
given.
Signed-off-by: Yuan Kang <Yuan.Kang@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add a PDB header file to support building protocol descriptors.
Signed-off-by: Steve Cornelius <sec@pobox.com>
Signed-off-by: Hemant Agrawal <hemant@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
init_desc, by always ORing with 1 for the descriptor header inclusion
into the descriptor length, and init_sh_desc_pdb, by always specifying
the descriptor length modification for the PDB via options, would not
allow for odd length PDBs to be embedded in the constructed descriptor
length. Fix this by simply changing the OR to an addition.
also round-up pdb_bytes to the next SEC command unit size, to
allow for, e.g., optional packet header bytes that aren't a
multiple of CAAM_CMD_SZ.
Reported-by: Radu-Andrei BULIE <radu.bulie@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Cc: Yashpal Dutta <yashpal.dutta@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In case of protocol acceleration descriptors, Shared descriptor header must
carry size of header length + PDB length in words which will be skipped by
DECO while processing descriptor to provide first command word offset
Signed-off-by: Yashpal Dutta <yashpal.dutta@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
SEC4 h/w gets configured in 32- vs. 36-bit physical
addressing modes depending on the size of dma_addr_t,
which is not always equal to sizeof(u32 *).
Also fixed alignment of a dma_unmap call whilst in there.
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Horia Geantă
Russell King
Alex Porosanu
Horia Geant?
Steve Cornelius
Vakul Garg
Linus Torvalds
Kim Phillips
Masanari Iida
Paul Bolle
Greg Kroah-Hartman
Ben Collins
Shengzhou Liu
Alex Porosanu
Bharat Bhushan
Yuan Kang
Hemant Agrawal
Yashpal Dutta