We've switched over every architecture that supports SMP to it, so
remove the new useless config variable.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Cc: Jan Kara <jack@suse.cz>
Cc: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Git-commit: 0a06ff068f1255bcd7965ab07bc0f4adc3eb639a
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[imaund@codeaurora.org: resolve merge conflicts]
Signed-off-by: Ian Maund <imaund@codeaurora.org>
Add a define for the test bio size (which is the size of a page),
this is used for allocating the right sized buffer for the bio during
test request creation.
Change-Id: I9505c85c4352009bdee442172eb8ae8f4254cfb0
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
dm-crypt provides bios based device mapper module. dm-crypt
operates on packets with 512 bytes size which is not effiicent
way for HW based crypto blocks. dm-req-crypt is developed to
address this. dm-req-crypt works on requests which carry upto
512KB of data for unmerged requests.
Change-Id: I7d6a63d516dc2dbe80f46c06dd0722847d55bc9f
Signed-off-by: Dinesh K Garg <dineshg@codeaurora.org>
MMC device driver implements URGENT request execution with priority
(using stop flow), as a result currently running (and prepeared) request
may be reinserted back into I/O scheduler. This will break block layer
logic of flushes (flush request should not be inserted into I/O scheduler).
Block layer flush machinery keep q->flush_data_in_flight list updated with
started but not completed flush requests with data (REQ_FUA).
This change will not notify underling block device driver about pending
urgent request during flushes in flight.
Change-Id: I98113621223fe0c7d224de023db888a73bd62b48
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
Update logging with:
- prefix with module name
- add '\n' in the end
- test_pr_* removed
Change-Id: I465c9809def9d294dcbb3f7cf7f474c189f5fdbf
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
This reverts commit f97d4f6148.
Although this check prevents NULL reference, it hides the real problem.
Requests that wish to avoid statistics update have to disable the
REQ_IO_STAT flag, otherwise req->part is expected to be initialized.
Change-Id: I680b95ab9aa668612d948770347929ffde30aeab
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
The flag REQ_IO_STAT is enabled by default this assumes statistics are
initialized and might cause NULL references in the kernel. To avoid it
this flag is cleared in the request and stats are not updated.
Change-Id: I6a1890dde51dfa8ffdd376b13f4466c9db0ae05b
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Block layer is accessing req->part without checking for NULL
pointer access, it is done when statistcs is not fully
initialized. Preventing the NULL pointer access effects only
the statistics update.
Change-Id: I45c91c074ecec1c3849f4f36185edcc6db35383c
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
The test will verify correctness of sequential data pattern
written to the device while new data (with same pattern) is
written simultaneously.
First this test will run a long sequential write scenario.
This first stage will write the pattern that will be read
later. Second, sequential read requests will read and
compare the same data. The second stage reads, will issue in
Parallel to write requests with the same LBA and size.
NOTE: The test requires a long timeout.
The purpose of this test is to mix read and write requests on the same
LBA while checking for the read data correctness.
Change-Id: I6a437ce689b66233af3055d07a7f62f1e7b40765
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Introduce a new callback 'check_test_completion_fn' to test-iosched
framework. This callback is necessary to determine if a test has
completed or not in situation where the request queue is empty, but the
test was not completed.
Change-Id: I60bd8cccffacab11a5a7cba78caccf53fea3e1d8
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Some times even though the block device is suspended by the block layer
the low-level driver might want to queue the PM requests to the device.
Allow such requests to get peeked as the blk_pm_add_request() has already
added it to the I/O scheduler otherwise the request would be forever stuck
in the I/O scheduler without being fetched by the driver.
Change-Id: I353943a7008ea1d92ff825d220cad1828fe37c27
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
This reverts commit b410a82118cdaa1dc92759e7995c20dcce0d1f1a.
The reverted commit was identified as the cause of the FS error
mentioned in the CR bellow.
It's reverted till further annalists of the root cause of FS error.
Change-Id: Ia75216de8012a2491b87f33e8c21f75592d87c80
CRs-fixed: 531257
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
When the scheduler reports to the block layer that there is an urgent
request pending, the device driver may decide to stop the transmission
of the current request in order to handle the urgent one. This is done
in order to reduce the latency of an urgent request. For example:
long WRITE may be stopped to handle an urgent READ.
Change-Id: I3072b8a1423870fed9c04c28d93caaf9557a7b89
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
This test adds the ability to test the UFS task management feature
in the driver. It loads the queue with requests in order to allow
the task management to operate in full capacity.
Modify test-iosched infrastructure to support the new tests:
- expose check_test_completion()
Note: we submit 16-bio requests since the current HW is very slow
and we don't want to exceed the timeout duration.
Change-Id: I8ee752cba3c6838d8edc05747fa0288c4b347ef6
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
Change time measurements in long_sequential_test from jiffies to ktime,
and make the relevant change in test-iosched infrastructure.
In long_sequential_test we measure throughput, and the jiffies resolution
is not sensitive enough for this calculation.
Change-Id: If7c9a03c687f61996609c014e056bcd7132b9012
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
It is possible for URGENT request to be requeued/reinserted if it was
fetched during the creation of a packed list. This end case is rare and is
not handled at the moment.
This patch changes the messages notifying of the above to debug level
(instead of error) in order to clear the dmesg log.
Change-Id: Ie8bc067e61559a6f702077b95c5dbcc426404232
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
There are cases when blk_peek_request is called not from blk_fetch_request
thus the URGENT request may be started but the flag q->dispatched_urgent is
not updated.
Change-Id: I4fb588823f1b2949160cbd3907f4729767932e12
CRs-fixed: 471736
CRs-fixed: 473036
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
An urgent request is marked by the scheduler in rq->cmd_flags with the
REQ_URGENT flag. There is no need to add an additional marking by
the block layer.
Change-Id: I05d5e9539d2f6c1bfa80240b0671db197a5d3b3f
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
The __blk_run_queue function is called from several contexts. The fix is
replacing it with blk_run_queue function, this function is guarded with
a lock, thus making it thread safe and prevents the crashing.
Change-Id: I3e12fa9c8b9e161375fffa3570abfa46b223a60b
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Change the test design so that requests are dynamically created
and freed. This enables running tests with more than 128 requests,
therefore more than 50MiB can be written/read and makes it possible
to measure driver write/read throughput more accurately.
Change-Id: I56c9d6c1afba5c91a0621a16d97feafd4689521d
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
Some block devices requires the rq_disk field to be assigned.
This patch exposes a new API to the block device test utility for
getting the rq_disk assigned, in the created request.
Change-Id: I61dc4dad50eb7600728156a6cd08bb1ee134df0d
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
The new request notification test checks the following scenario:
A new request arrives after a NULL request was sent to the mmc_queue,
which is waiting for completion of a former request.
Change-Id: I05db0959ded400e292eb5e84e1ecfc579b78ee62
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
Add functionality to test-iosched so that it could simulate the
ROW scheduler behaviour. The main additions are:
- 3 distinct requests queue with counters
- support for urgent request pending
- reinsert request implementation (callback + dispatch behavior)
Change-Id: I83b5d9e3d2b8cd9a2353afa6a3e6a4cbc83b0cd4
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
Enable the compilation of eMMC4.5 unit-tests, required by APT team.
This will allow the APT team to test the storage activity on released
builds.
The storage tests are disabled in normal operation and in order to
activate them a test I/O scheduler should be chosen and the test should
be triggered via debugfs. Therefore they have no effect on normal eMMC
driver operation.
Change-Id: I179c567f67cc8fab9ed1edab8246483de18bc76a
Signed-off-by: Maya Erez <merez@codeaurora.org>
Change the printout format to be more readable.
Specifically, add quotes around the test case name
strings.
Change-Id: I51b0c1b94389e4b51af84c5e993207b18efc2226
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
Long sequential read test measures read throughput
at the driver level by reading large requests sequentially.
Change-Id: I3b6d685930e1d0faceabbc7d20489111734cc9d4
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
* qandroid-3.10: (636 commits)
netfilter: xt_qtaguid: Protect iface list access with necessary lock
HID: magicmouse: Fix build warning
USB: gadget: mtp: Fix OUT endpoint request length usage in read
USB: gadget: f_mtp: Fix using tx buffer pointer
msm: Fix race condition in domain lookup
msm: Add null-pointer checks for domains
base: sync: increase size of sync_timeline name
USB: gadget: mtp: Add module parameters for Tx transfer length
msm: iommu: Lock the genpool allocation
gpu: ion: fix page offset in dma_buf_kmap()
gpu: ion: Fix bug in ion_system_heap map_user
gpu: ion: Only map as much of the vma as the user requested
gpu: ion: use vmalloc to allocate page array to map kernel
gpu: ion: Remove dead comments
gpu: ion: Minimize allocation fallback delay
mmc: sd: Set the card removed if card detect fails
gpu: ion: don't fault in individual pages for the CP heap
gpu: ion: do not ask for compound pages in system heap
gpu: ion: Modify the system heap to try to allocate large/huge pages
gpu: ion: Set the dma_address of the sg list at alloc time
...
Conflicts:
arch/arm/Kconfig
arch/arm/include/asm/hardware/cache-l2x0.h
arch/arm/mm/cache-l2x0.c
drivers/mmc/card/block.c
drivers/usb/gadget/udc-core.c
Regular priority queues is marked as "starved" if it skipped a dispatch
due to being empty. When a new request is added to a "starved" queue
it will be marked as urgent.
The removed WARN_ON was warning about an impossible case when a regular
priority (read) queue was marked as starved but wasn't empty. This is
a possible case due to the bellow:
If the device driver fetched a read request that is pending for
transmission and an URGENT request arrives, the fetched read will be
reinserted back to the scheduler. Its possible that the queue it will be
reinserted to was marked as "starved" in the meanwhile due to being empty.
CRs-fixed: 517800
Change-Id: Iaae642ea0ed9c817c41745b0e8ae2217cc684f0c
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
Calling hrtimer_cancel with interrupts disabled can result in a livelock.
When flushing plug list in the block layer interrupts are disabled and an
hrtimer is used when adding requests from that plug list to the scheduler.
In this code flow, if the hrtimer (which is used for idling) is set, it's
being canceled by calling hrtimer_cancel. hrtimer_cancel will perform
the following in an endless loop:
1. try cancel the timer
2. if fails - rest_cpu
the cancellation can fail if the timer function already started. Since
interrupts are disabled it can never complete.
This patch reduced the number of times the hrtimer lock is taken while
interrupts are disabled by calling hrtimer_try_co_cancel. the later will
try to cancel the timer just once and return with an error code if fails.
CRs-fixed: 499887
Change-Id: I25f79c357426d72ad67c261ce7cb503ae97dc7b9
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
In the current implementation idling is triggered only by request
insertion frequency. This heuristic is not very accurate and may hit
random requests that shouldn't trigger idling. This patch uses the
PG_readahead flag in struct page's flags, which indicates that the page
is part of a readahead window, to start idling upon dispatch of a request
associated with a readahead page.
The above readehead flag is used together with the existing
insertion-frequency trigger. The frequency timer will catch read requests
which are not part of a readahead window, but are still part of a
sequential stream (and therefore dispatched in small time intervals).
Change-Id: Icb7145199c007408de3f267645ccb842e051fd00
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
The block layer implements a mechanism for verifying that the device
driver won't be notified of an URGENT request if there is already an
URGENT request in flight. This is due to the fact that interrupting an
URGENT request isn't efficient.
This patch fixes the above described mechanism in case the URGENT request
was returned back to the block layer from some reason: by requeue or
reinsert.
CRs-fixed: 473376, 473036, 471736
Change-Id: Ie8b8208230a302d4526068531616984825f1050d
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
The current starvation tolerance values increase the boot time
since high priority SW requests are delayed by regular priority requests.
In order to overcome this, increase the starvation tolerance values.
Change-Id: I9947fca9927cbd39a1d41d4bd87069df679d3103
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
Signed-off-by: Maya Erez <merez@codeaurora.org>
All ROW (time related) configurable parameters are stored in ms so there
is no need to convert from/to ms when reading/updating them via sysfs.
Change-Id: Ib6a1de54140b5d25696743da944c076dd6fc02ae
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
At the moment all REGULAR and LOW priority requests are starved as long as
there are HIGH priority requests to dispatch.
This patch prevents the above starvation by setting a starvation limit the
REGULAR\LOW priority requests can tolerate.
Change-Id: Ibe24207982c2c55d75c0b0230f67e013d1106017
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
When ROW scheduler reports to the block layer that there is an urgent
request pending, the device driver may decide to stop the transmission
of the current request in order to handle the urgent one. This is done
in order to reduce the latency of an urgent request. For example:
long WRITE may be stopped to handle an urgent READ.
This patch updates the ROW URGENT notification policy to apply with the
below:
- Don't notify URGENT if there is an un-completed URGENT request in driver
- After notifying that URGENT request is present, the next request
dispatched is the URGENT one.
- At every given moment only 1 request can be marked as URGENT.
Independent of it's location (driver or scheduler)
Other changes to URGENT policy:
- Only READ queues are allowed to notify of an URGENT request pending.
CR fix:
If a pending urgent request (A) gets merged with another request (B)
A is removed from scheduler queue but is not removed from
rd->pending_urgent_rq.
CRs-Fixed: 453712
Change-Id: I321e8cf58e12a05b82edd2a03f52fcce7bc9a900
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
In order to be sure that the packing statistics collected after the test
reflect *only* requests issued by the test (and not real request from
FS) - sleep before each test in order to give an already dispatched
requests time to complete.
Change-Id: If2f40efad1d79084a8ea85afe93cce58e49ff698
CRs-Fixed: 453712
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
This patch sets the initial values of internal ROW
parameters.
Change-Id: I38132062a7fcbe2e58b9cc757e55caac64d013dc
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
[smuckle@codeaurora.org: ported from msm-3.7]
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
When ROW scheduler reports to the block layer that there is an urgent
request pending, the device driver may decide to stop the transmission
of the current request in order to handle the urgent one. If the current
transmitted request is an urgent request - we don't want it to be
stopped.
Due to the above ROW scheduler won't notify of an urgent request if
there are urgent requests in flight.
Change-Id: I2fa186d911b908ec7611682b378b9cdc48637ac7
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
At the moment idling in ROW is implemented by delayed work that uses
jiffies granularity which is not very accurate. This patch replaces
current idling mechanism implementation with hrtime API, which gives
nanosecond resolution (instead of jiffies).
Change-Id: I86c7b1776d035e1d81571894b300228c8b8f2d92
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
This patch implements "application-hints" which is a way the issuing
application can notify the scheduler on the priority of its request.
This is done by setting the io-priority of the request.
This patch reuses an already existing mechanism of io-priorities developed
for CFQ. Please refer to kernel/Documentation/block/ioprio.txt for
usage example and explanations.
Change-Id: I228ec8e52161b424242bb7bb133418dc8b73925a
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
Each ROW queues has several parameters which default values are defined
in separate arrays. This patch aggregates all default values into one
array.
The values in question are:
- is idling enabled for the queue
- queue quantum
- can the queue notify on urgent request
Change-Id: I3821b0a042542295069b340406a16b1000873ec6
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
idle_time was updated to be stored in msec instead of jiffies.
So there is no need to convert the value when reading from user or
displaying the value to him.
Change-Id: I58e074b204e90a90536d32199ac668112966e9cf
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
There is really no point in keeping the dispatch quantum
of a queue outside of it. By inserting it to the row_queue
structure we spare extra level in accessing it.
Change-Id: Ic77571818b643e71f9aafbb2ca93d0a92158b199
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
1. Add a counter for number of requests on queue.
2. Add function to print queues status (number requests
currently on queue and number of already dispatched requests
in current dispatch cycle).
Change-Id: I1e98b9ca33853e6e6a8ddc53240f6cd6981e6024
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
This patch adds support for handling urgent requests.
ROW queue can be marked as "urgent" so if it was un-served in last
dispatch cycle and a request was added to it - it will trigger
issuing an urgent-request-notification to the block device driver.
The block device driver may choose at stop the transmission of current
ongoing request to handle the urgent one. Foe example: long WRITE may
be stopped to handle an urgent READ. This decreases READ latency.
Change-Id: I84954c13f5e3b1b5caeadc9fe1f9aa21208cb35e
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
Add support for reinserting already dispatched request back to the
schedulers internal data structures.
The request will be reinserted back to the queue (head) it was
dispatched from as if it was never dispatched.
Change-Id: I70954df300774409c25b5821465fb3aa33d8feb5
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
NULL pointer dereference can happen in cfq_choose_cfqg()
when there are no cfq groups to select other than the
current serving group. Prevent this by adding a NULL
check before dereferencing.
Unable to handle kernel NULL pointer dereference at virtual address
[<c02502cc>] (cfq_dispatch_requests+0x368/0x8c0) from
[<c0243f30>] (blk_peek_request+0x220/0x25c)
[<c0243f30>] (blk_peek_request+0x220/0x25c) from
[<c0243f74>] (blk_fetch_request+0x8/0x1c)
[<c0243f74>] (blk_fetch_request+0x8/0x1c) from
[<c041cedc>] (mmc_queue_thread+0x58/0x120)
[<c041cedc>] (mmc_queue_thread+0x58/0x120) from
[<c00ad310>] (kthread+0x84/0x90)
[<c00ad310>] (kthread+0x84/0x90) from
[<c000eeac>] (kernel_thread_exit+0x0/0x8)
CRs-Fixed: 416466
Change-Id: I1fab93a4334b53e1d7c5dcc8f93cff174bae0d5e
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
This patch addresses the following issues found in the ROW idling
mechanism:
1. Fix the delay passed to queue_delayed_work (pass actual delay
and not the time when to start the work)
2. Change the idle time and the idling-trigger frequency to be
HZ dependent (instead of using msec_to_jiffies())
3. Destroy idle_workqueue() in queue_exit
Change-Id: If86513ad6b4be44fb7a860f29bd2127197d8d5bf
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
This patch add support in block & elevator layers for handling
urgent requests. The decision if a request is urgent or not is taken
by the scheduler. Urgent request notification is passed to the underlying
block device driver (eMMC for example). Block device driver may decide to
interrupt the currently running low priority request to serve the new
urgent request. By doing so READ latency is greatly reduced in read&write
collision scenarios.
Note that if the current scheduler doesn't implement the urgent request
mechanism, this code path is never activated.
Change-Id: I8aa74b9b45c0d3a2221bd4e82ea76eb4103e7cfa
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
Add support for reinserting a dispatched request back to the
scheduler's internal data structures.
This capability is used by the device driver when it chooses to
interrupt the current request transmission and execute another (more
urgent) pending request. For example: interrupting long write in order
to handle pending read. The device driver re-inserts the
remaining write request back to the scheduler, to be rescheduled
for transmission later on.
Add API for verifying whether the current scheduler
supports reinserting requests mechanism. If reinsert mechanism isn't
supported by the scheduler, this code path will never be activated.
Change-Id: I5c982a66b651ebf544aae60063ac8a340d79e67f
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
