When we map ion buffers as uncached we want to map them as
uncached but still be able to use buffering of transactions
since turning off buffering has a severe performance impact.
Change-Id: I5a09ac34bd3ce3f001d10d9efd71d140f632564d
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
When allocating pages for IOMMU heap it is advantageous
to try to allocate pages from highmem in addition to
lowmem since clients using the IOMMU heap can use
either highmem or lowmem.
CRs-fixed: 430457
Change-Id: Ifc794fc344c67dd53d8f7cebf86536ab9d44ed6c
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
When allocating pages that are intended to be used for
uncached allocations, we need to ensure the cache is coherent;
there may be outstanding data in the cache related to those
pages. Ensure cache coherency by flushing each of the pages.
Change-Id: I4b89c799b5c099f6c050d8ddd758bdb368c07c08
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
On certain targets, the CP heap should only be used
for secure allocations. Add a check to determine which
targets are allowed to make non-secure allocations from
the CP heap type. Targets with this restriction will
fall back to an alternate heap.
Change-Id: Ieaa9e76cbf2dc3ea1da6f4e75a4de903c39a3077
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Extend the cp heap to allow memory to be allocated from
the contiguous memory allocator (CMA) instead of from
the standard caveout region. The option to use CMA or regular
carveout memory is configured via a parameter in platform
data.
Change-Id: I9f3a169325c44230dde1d91a9cdcf613ad291df2
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
request_region and release_region are macro names defined
in the linux kernel. Under some circumstances, the C compiler
can't differentiate between the macro name and the field name.
Changing the field name is the easiest way to prevent this
problem.
Change-Id: I2c8d61bdaa20e332e0215f0bb3237e8332f0f3ac
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Currently, fmem must be transitioned on first allocation/
last free. Going forward, there may be other use cases to
call functions on first allocation/last free. Factor some of
this code out to avoid duplication.
Change-Id: I36472333222c497c5b4c888394b4bd277c146249
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
A number of changes have been made to the Ion framework for the
msm target. Add the necessary changes on top of the CMA heap to
allow the CMA heap to be fully utilized.
Change-Id: Ie006dcd4c41481e4d914c67bafbf42d1afdb1a76
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
New heap type ION_HEAP_TYPE_DMA where allocation is done with dma_alloc_coherent API.
device coherent_dma_mask must be set to DMA_BIT_MASK(32).
ion_platform_heap private field is used to retrieve the device linked to CMA,
if NULL the default CMA area is used.
ion_cma_get_sgtable is a copy of dma_common_get_sgtable function which should
be in kernel 3.5
Change-Id: I9ae54a3a021cb3513c2b0e8c58b69f3ae118561b
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@linaro.org>
[lauraa: Fix context in ion_priv.h/ion.h and omit Makefile change for now]
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
copy private field from platform configuration to internal heap structure.
Change-Id: Ia7571d88fc2f72f5d655fb6f6b54fde389d96c85
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@linaro.org>
[laura: Rebase context fixes]
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
This is another step in the process of isolating msm-specific ion
features from stock ion.
Change-Id: I3a437dbc618cb70859126c81596373338ad06500
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Buffer flags are going to be specified at allocation time rather than
map time. This removes the flags argument from the ion kernel map
function.
Change-Id: Ib983ecd0dcd7befb36287ae7037c71d4ca475f90
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Heap mask field passes as argument at allocation time to specify ion
heap.
Change-Id: I6951da81b11bfcb0e006d2539c1707aabbb12db6
Signed-off-by: Hanumant Singh <hanumant@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
This syncs up our ion buffer flags in ion.h with upstream (using
ION_FLAG_CACHED instead of CACHED and UNCACHED).
Change-Id: I2e07a3acba230a86817182677fa2394fa846fd7e
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
ion_validate_buffer_flag is no longer needed since its only callers
are moving to a new format for buffer flags.
Change-Id: I90f23123681eaefb40b530aec5eaf596770acf18
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
To avoid collisions with future ioctls that may be added to ion,
msm-specific ioctls should use the `custom_ioctl' interface provided
by struct ion_device, as originally intended.
Change-Id: I68d32b81c7691fd3f54f38acfdc17fee2573bc44
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
All msm_ion clients need to use <linux/msm_ion.h> instead of
<linux/ion.h>
Change-Id: I4f999e301d147ec3242a33ee75b10b6501d42a70
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
If preemted during ion_free after the refcount is updated but
before the handle can be removed from the rb_tree, import
might find that handle in the tree and try to reuse it
when execution returns to free, the handle will be cleaned
up leaving the caller of import with a corrupt handle.
This patch modifies the locking to protect agains this race.
Change-Id: I31d18cc6398f0ca18e05cd919e2bcf86fa18d568
CRs-Fixed: 385283
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
[lauraa@codeaurora.org: Whitespace change and move unlock]
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Ajay Dudani <adudani@codeaurora.org>
Add device tree parsing to common msm ion code
to allow new targets to specify ION heap configuration
in device tree.
Document the device tree bindings for ION.
Change-Id: I40b7f4b1fbb3ca4ad1d6dc06a33b647f6af1d669
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
If dma_buf_fd fails, the dma_buf needs to be cleaned up by
calling dma_buf_put. dma_buf_put will call ion_dma_buf_release
which in turn calls ion_buffer_put to clean up the buffer
reference. Calling ion_buffer_put after dma_buf_put drops the
reference count by one more which is incorrect. Fix this by
getting rid of the extra ion_buffer_put call.
Change-Id: Ieb3a7b781bf2dc299c45384dbf0a12d186adeb22
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The condition argument to the WARN call in ion_share_dma_buf
is missing. Add the argument to allow correct printing of
warning message.
Change-Id: I6ded3a78d3030f38bf29a1be063364690cbc0ddb
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Add EXPORT_SYMBOL to specific functions that should be
allowed to be used from within kernel modules.
Change-Id: I666c1f1a18d5af58906cb2185c5cf29d8596013f
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Add support to allow ion_cp_heap.c to secure according to new
requirements. This includes adding versioning to the secure
calls to allow other types of securing to take place.
Change-Id: I65e07ebaeefa1d0572b6531753a707a28284aa0d
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
In the IOMMU heap, the return value of msm_allocate_iova_address
needs to be checked instead of the value of the iova address.
Check the return value instead.
Change-Id: I4554515d17c926a7dd738824aff21eb65d2ba56a
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Reference count the calls to ion_{un}secure_heap. The secure SCM
call will only be made when the count goes from 0 -> 1 and the
unsecure call will only be made when the count goes from 1 -> 0.
Change-Id: Ia221b79782c9223d9853ef00c2848e1ab7a73091
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The next generation of content protection has separate SCM
calls into TZ. Add these functions in a dedicated file so they
can be used by any Ion heap.
Change-Id: I197130f6a5fc33efcd6871b0f7c362fa99f18d20
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Moves fixed area start address down, so that non secure
video firmware heap and secure video firmware area overlap.
Both use the region in a mutually exclusive manner.
ALso introduce a hole, that can be used by MM heap
lock down for content protection.
Change-Id: Ic240377da25e00423742ccb73dbb60a1ff037e37
Signed-off-by: Deepak Kotur <dkotur@codeaurora.org>
[sboyd: drop board file changes]
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Instead of using custom routines for cache operations use the
standard operations for cache flush, cache clean, and cache
invalidate. In addition only call outer cache routines if
outer cache is available.
Change-Id: Idb51bf7348fd526ebdea1e6ff2e366b8a3b4de33
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Going forward, clients may want to have more control over the
virtual address space for iommu domains. Add API to support
clients registering virtual address space information. Current
clients are internally converted to use the new APIs.
Change-Id: Iaec9f1cd150eefc02ce22c34790494b61f21eb12
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Add dump of physical memory map, showing allocated and
free areas, to debugfs interface for heaps with carveout memory.
Change-Id: I9bda9f3e555e55570c95e652616ca1fcc25eb0ab
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Access to rb tree of user and kernel ION clients
must be protected by a mutex.
Change-Id: Ie27eba65f8cab3a7c20c041eb46429ba62c5c00f
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
When the last buffer is freed from a heap that uses fmem we call
into fmem to transition the memory from c-state to t-state. However,
if the heap has been secured for content protection transitioning
the memory from c-state to t-state could cause undefined behavior
in fmem.
Add checks to ensure that a secured heap will not transition to
t-state until heap is unsecured.
Change-Id: I672274c8bcb9912652e82588941fe840b08b3473
CRs-fixed: 359593
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Add logging of buffer physical address (if allocation
is from physically contiguous heap) and iommu mapping
information to the ion client debug interface.
Change-Id: Id3a77c1b5e3c5d15b11b3612f07d10092922cdff
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Add check for null argument to avoid dereferencing
a null-pointer.
Change-Id: Icb452d0764e375bf2684e013c51d260de98e8583
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
ION_IOC_MAP, ION_IOC_SHARE, and ION_IOC_IMPORT may return
success when an error occurs.
Add correct error handling to ION_IOC_MAP, ION_IOC_SHARE, and
ION_IOC_IMPORT.
Change-Id: Idb7e58c94dab53fd82dd49150af9eaec78430c40
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
During bootup add check to ensure heap addresses are not overlapping
with each other. If a heap is found to be overlapping with another
heap boot will be stopped with a call to panic.
Change-Id: Id37aefa2e09aa165ec376e241e282faf5f3c5737
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Under certain error conditions the following ION ioctl's would return
success even when the operation failed:
-ION_IOC_CLEAN_CACHES
-ION_IOC_INV_CACHES
-ION_IOC_CLEAN_INV_CACHES
Add return of correct error codes when an error occurs.
Change-Id: Ie8824bfecbb423c23d32c6a3d7058f06d8322906
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Due to performance issues with 4K pages and the IOMMU we should use
64K pages. However, minimum allocation from ION is 4K so instead of
mapping on demand we need to map the full heap into the IOMMU when
the first request for mapping comes. Only unmap everything from the
IOMMU when the last buffer is freed.
CRs-fixed: 348606
Change-Id: Ic1793f5caaff2f69bf1fb7e8c8b3bd03378131b8
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
[sboyd: drop board file, fmem, memory changes]
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
IOMMU virtual address (iova) is allocated locally in IOMMU mapping
functions per heap type. When an error happens the iova's are
freed in the same function as it was allocated. However, there
is an extra free call in the ion wrapper function that calls
these IOMMU mapping functions.
Remove the extra iova free call which can cause kernel panic.
Change-Id: Ic8978cf05202b29a42da7374e2c8c1e590b2d2a8
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
iommu map functions requires clients to specify whether
the mappings should be read only or read-write (write only
is not supported). Ensure clients specify the appropriate
flags when calling map functions.
Change-Id: If42b4b0367b4311aa36d3063979ef04af3a43fc6
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
[sboyd: drop kgsl changes]
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
The condition argument to the WARN call in ion_free is missing. Add the
argument to allow correct printing of warning message.
Change-Id: I158e1a583b7242b2c3209d6ce64dc0b771efeffe
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
If an ion allocation fails because of memory fragmentation a debug
message should be printed. However, the logic that decides whether
the debug message should be printed or not is incorrect causing
the debug message to not be printed when the amount of space
available in the heap is equal to the requested size.
Change-Id: Iaaf8cd6742aeffab7bcb4806e485326fb9690945
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
If an ion allocation fails because of memory fragmentation a debug
message should be printed. However, the logic that decides whether
the debug message should be printed or not is incorrect causing
the debug message to not be printed when it should.
Change-Id: I6537171db0a47ba072d182e9bb3125cac5513ff9
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
The multimedia firmware on 8960 will always be
loaded into a fixed address, and all callers into
this firmware must be no more than 256M above the
start of this firmware.
This implies that the the mm ION heap and its associated heaps
(mm_fw and mfc) must also be created in a fixed place.
This version of the patch keeps the mm fw heap size
to 2M for compatibility with the current tip. This
will eventually be reduced to 1M.
Change-Id: I35f89b8c60165fb0adbb9d04412e0e200117beb9
Signed-off-by: Larry Bassel <lbassel@codeaurora.org>
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
[sboyd: drop memory.c changes]
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Instead of mapping 1 4K page at a time into the IOMMU create a
scatterlist and map everything at once. This will be more efficient.
Change-Id: I8e83066869dd6f7a479bad22a66e4c70cc5973b5
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Ion carveout and content protect heap buffers do not
have a struct page associated with them. Thus
sg_phys() will not work reliably on these buffers,
so set dma_address on their scatterlists.
CRs-Fixed: 345257
Change-Id: Ifdad5ce497de170f47b4ee2f7a93563a5cbe1a96
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Due to limitations in the multimedia architecure
clients might not know when a buffer can be unmapped
from the IOMMU. In addition, the multimedia architecture
causes unnecessary mappings/unmappings for the same buffers
which reduces framerates.
Add logic to delay unmapping from the IOMMU
until buffer memory is freed and unmap any outstanding
mappings to avoid virtual memory address space leak.
Change-Id: Idaeae269d9ba623e25a0cb087a89b4cbb63915af
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
IOMMU is missing support for ion_map_dma.
ION clients need support for this api.
Add functions to implement ion_map_dma and
ion_unmap_dma in the IOMMU heap.
Change-Id: If5723f0d7ec1145e9c9d8b88a42b57ab0f2c2da7
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
The IOMMU heap must support cache operations so that clients
can do cached allocations from the heap.
Add function to perform cache operations on memory
allocated from IOMMU heap.
Change-Id: I49a84dca613b7c209dde03b67b2fc1b2c9d844dd
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Clients using ION must be able to mmap a smaller
set of pages than what was allocated from the
IOMMU heap.
Add check that we don't try to map a page passed
the end of the vma.
Change-Id: I1241f952359f6bb1d5bb0deb11cc0f44d94a3404
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
