-
Notifications
You must be signed in to change notification settings - Fork 200
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Replace DispatchQueue and BookingQueue with HealthyThreadPool (#1035)
* Update dispatchQuery to use min_cores Sorting jobs only by priority causes a situation where low priority jobs can get starved by a constant flow of high priority jobs. The new formula adds a modifier to the sorting rank to take into account the number of cores the job is requesting and also the number of days the job is waiting on the queue. Priorities numbers over 200 will mostly override the formula and work as a priority only based scheduling. sort = priority + (100 * (1 - (job.cores/job.int_min_cores))) + (age in days) Besides that, also take layer_int_cores_min into account when filtering folder_resourse limitations to avoid allocating more cores than the folder limits. (cherry picked from commit 566411aeeddc60983a30eabe121fd03263d05525) * Revert "Update dispatchQuery to use min_cores" This reverts commit 2eb4936 * Replace DispatchQueue and BookingQueue with HealthyThreadPool Queues will not inherit from ThreadPoolExecutor, instead they will manage an instance of HealthThreadPool, which is a threadPoolExecutor that handles healthChecks, termination and repeated tasks. With this the Booking queue should be able to self-heal when locked threads happen. * Remove trackit reference * Refactor HostReportQueue to use guava Cache Use a guava cache to store only the last version of a HostReport per host. * Configure HostReportQueue on opencue.properties * Fix unit tests * Fix unit tests * This unit tests is not actually testing anything useful Test doesn't make sense with the new threadpool and will also cause problems whenever an user changes a config property. Co-authored-by: Roula O'Regan <roregan@imageworks.com>
- Loading branch information
1 parent
6da47ba
commit 027d853
Showing
37 changed files
with
781 additions
and
363 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.