Commit graph

104 commits

Author SHA1 Message Date
Prinz23
6da32a5ed0 Add log message for not found on indexer when adding a new show.
Fix upgrade once ARCHIVED setting by postProcessor.
Fix determination of is_first_best_match.
Change improve smart selection of categories in manual and failed search modes.
Change refactor wantedQuality into own function that can be used in multiple places.
Change improve error resistance in neededQualities class.
Add log warning message if wantedQuality or eps_aired_in_season is missing for search.
Add check backlogitem for wantedQuality and add if missing.
Add use wantedQuality list in wantEpisode.
Change don't use wantedQualities for multipart.
2017-12-27 03:14:20 +00:00
Prinz23
05758bb75e Change overhaul handling of PROPERS/REPACKS/REAL.
Add restriction to allow only same release group for repacks.
Change try all episode names with 'real', 'repack', 'proper'.
Add flags table to store when one-off operations have been performed.
Change update history.tbl with action SNATCHED_PROPER on existing items.
Change save proper snatches as action SNATCHED_PROPER + quality into history.tbl
Add optional regex lib with fuzzy (errors ~20% of alpanum chars) for extra_info_no_name.
Add tip to search settings/media search about improved matching with optional regex library.
2017-10-30 22:21:55 +00:00
Prinz23
33a36432d8 Change improve newnab autoselect categories.
Change add nzb.org BoxSD and BoxHD categories.
Change improve category exclude logic for newznab.
Change add WEB-DL category to autoselect categories for newznab.
Change comment out HEVC category parsing since it's not auto selected.
2017-08-19 15:53:59 +01:00
Prinz23
266c87f5f1 Change catch show update task errors.
Change tracebacks to ERROR.
Add scene_exceptions error handling with missing data.
Add exception catching and traceback logging in show_updater.
2017-06-15 01:37:28 +01:00
JackDandy
fc5e0df46c Merge branch 'master' into develop 2017-02-17 05:11:53 +00:00
JackDandy
d97bb7174d Remove Torrentshack.
Change improve provider title processing.
Change improve handling erroneous JSON responses.
Change improve find show with unicode characters.
Change improve result for providers Omgwtf, SpeedCD, Transmithenet, Zoogle.
Change validate .torrent files that contain optional header data.
Fix case where an episode status was not restored on failure.
Add raise log error if no wanted qualities are found.
Change add un/pw to Config/Media providers/Options for BTN API graceful fallback (can remove Api key for security).
Change only download torrent once when using blackhole.
Add Cloudflare module 1.6.8 (be0a536) to mitigate CloudFlare (IUAM) access validator.
Add Js2Py 0.43 (c1442f1) Cloudflare dependency.
Add pyjsparser 2.4.5 (cd5b829) Js2Py dependency.
2017-02-17 04:48:49 +00:00
JackDandy
14db6d12cc Add "Perform search tasks" to Config/Media Providers/Options.
Change improve clarity of enabled providers on Config/Media Providers.
2017-02-15 03:32:56 +00:00
Prinz23
87e608f277 Add scheduled backlog switch. 2017-02-15 00:51:52 +00:00
Prinz23
e071542b51 Change don't exclude "unknown" from search.
Fix UHD category select in Recent Search.
Change speed improvement in finding needed categories/qualities (sd, hd, uhd).
2016-11-14 01:50:25 +00:00
Prinz23
5adf5071d7 Change allow pp to replace files with a repack or proper of same quality.
Fix ensure downloaded eps are not shown on episode view.
Fix allow propers to pp when show marked upgrade once.
Fix never set episodes without airdate to wanted.
Various other fixes.
2016-10-01 11:27:25 +01:00
Prinz23
d3a7f0ff5e Add smart logic to reduce api hits to newznab server types and improve how nzbs are downloaded.
Add newznab smart logic to avoid missing releases when there are a great many recent releases.
Change improve performance by using newznab server advertised capabilities.
Change config/providers newznab to display only non-default categories.
Change use scene season for wanted segment in backlog if show is scene numbering.
Change combine Manage Searches / Backlog Search / Limited and Full to Force.
Change consolidate limited and full backlog.
Change config / Search / Backlog search frequency to instead spread backlog searches over a number of days.
Change migrate minimum used value for search frequency into new minimum 7 for search spread.
Change restrict nzb providers to 1 backlog batch run per day.
Add to Config/Search/Unaired episodes/Allow episodes that are released early.
Add to Config/Search/Unaired episodes/Use specific api requests to search for early episode releases.
Add use related ids for newznab searches to increase search efficiency.
Add periodic update of related show ids.
Change terminology Edit Show/"Post processing" tab name to "Other".
Add advanced feature "Related show IDs" to Edit Show/Other used for finding episodes and TV info.
Add search info source image links to those that have zero id under Edit Show/Other/"Related show IDs".
Add "set master" button to Edit Show/Other/"Related show IDs" for info source that can be changed.
Change terminology displayShow "Indexers" to "Links" to cover internal and web links.
Change add related show info sources on displayShow page.
Change don't display "temporarily" defunct TVRage image link on displayShow pages unless it is master info source.
Change if a defunct info source is the master of a show then present a link on displayShow to edit related show IDs.
Change simplify the next backlog search run time display in the page footer.
Change try ssl when fetching data thetvdb, imdb, trakt, scene exception.
Change improve reliability to Trakt notifier by using show related id support.
Change improve config/providers newznab categories layout.
Change show loaded log message at start up and include info source.
Change if episode has no airdate then set status to unaired (was skipped).

Technical
Change move scene_exceptions table from cache.db to sickbeard.db.
Add related ids to show obj.
Add use of mapped indexer ids for newznab.
Add indexer to sql in wanted_eps.
Add aired in (scene) season for wanted episodes.
Add need_anime, need_sports, need_sd, need_hd, need_uhd to wanted episodes and added as parameter to update_providers.
Add fix for lib lockfile/mkdirlockfile.
Add set master TV info source logic.
Change harden ui input validation.
Add per action dialog confirmation.
Change to reload page under more events.
Change implement "Mark all added episodes Wanted to search for releases" when setting new info source.
2016-09-21 17:50:27 +01:00
JackDandy
eac69963f0 Change validate and improve specific Torrent provider connections, IPT, KAT, SCC, TPB, TB, TD, TT.
Change refactor cache for torrent providers to reduce code.
Change improve search category selection BMTV, FSH, FF, TB.
Change identify more SD release qualities.
Change update SpeedCD, MoreThan, TVChaosuk.
Add torrent provider HD4Free.
Remove torrent provider BitSoup.
Change only create threads for providers needing a recent search instead of for all enabled.
Add 4489 as experimental value to "Recent search frequency" to use provider freqs instead of fixed width for all.
Fix searching nzb season packs.
Change remove some logging cruft.
2016-06-08 01:55:13 +01:00
Prinz23
6dd434fcba Add CPU throttling preset "Disabled" to config/General/Advanced Settings and centralise the function. 2016-02-11 19:25:35 +00:00
JackDandy
43778d8edd Change providers, add some, remove one, fix a snatch issue, rework ignore/require words, refactor code.
Add BitMeTV torrent provider.
Add TVChaosUK torrent provider.
Add HD-Space torrent provider.
Add Shazbat torrent provider.
Remove Animenzb provider.
Change use tvdbid for searching usenet providers.
Change consolidate global and per show ignore and require words functions.
Change "Require word" title and notes on Config Search page to properly describe its functional logic.
Add "Reject Blu-ray M2TS releases" to BTN provider.
Add regular expression capability to ignore and require words by starting wordlist with "regex:".
Add list shows with custom ignore and require words under the global counterparts on the Search Settings page.
Fix failure to search for more than one selected wanted episode.
2015-12-02 01:31:50 +00:00
JackDandy
2051268891 Change refactor, PEP8, code convention cleanse for nzb/torrent code and remove newzbin remnants. 2015-07-15 19:19:39 +01:00
Prinz23
2e711423b9 Add Search Queue Overview page
* Add expandable search queue details on the Manage Searches page
* Fix failed status episodes not included in next_episode search function
* Change prevent another show update from running if one is already running
* Change split Force backlog button on the Manage Searches page into: Force Limited, Force Full
* Change refactor properFinder to be part of the search
* Change improve threading of generic_queue, show_queue and search_queue
* Change disable the Force buttons on the Manage Searches page while a search is running
* Change disable the Pause buttons on the Manage Searches page if a search is not running
* Change scheduler forceRun
* Change staggered periods of testing and updating of all shows "ended" status up to 460 days
2015-05-10 17:22:54 +02:00
Prinz23
4ae30ab6ad Add option "Search for unaired episodes" to config/Search Settings
* Change reduce time to search recent result list by searching only once for a best result
* Fix replacing episodes that have a lower quality than what is selected in the initial and archive quality list
* Fix to include episodes marked Failed in the recent and backlog search processes
* Fix display of search status for an alternative release after episode is manually set to "Failed" on the Display Show page
* Change handle more varieties of media quality
* Change to prevent another scheduled search when one of the same type is already running
2015-04-24 20:45:45 +02:00
JackDandy
c96dbf7ea6 Change to reduce the time taken to "Update shows" with show data.
Add "Enable IMDb info" option to config/General/Interface.
Change to not display IMDb info on UI when "Enable IMDb info" is disabled.
Change genre tags on displayShow page to link to IMDb instead of Trakt.
Change to reduce the time taken to "Update shows" with show data.
Change when IMDb info is updated...
a) stop updating the IMDb info during the scheduled daily update for every show.
b) update the IMDb info for a show after snatching an episode for it.
Develop changes...
Fix being able to actually turn IMDb off when it has been turned on.
Remove IMDb option from General Settings.
Change IMDb option to enable by default now the slow operation has been eliminated from process flows.
2015-03-14 15:09:39 +00:00
JackDandy
e6b7feef3d Change to simplify and reduce logging output of Recent-Search and Backlog processes. 2015-03-13 18:21:46 +00:00
adam
d1687dd401 Change layout of Recent Search code
Change naming of SEARCHQUEUE threads for shorter log lines
Fix Recent Search running status on Manage Searches page
2015-03-12 15:31:35 +08:00
JackDandy
c16903bd73 Change "Daily" search to "Recent" search. 2015-01-07 19:54:18 +08:00
Supremicus
4c8c6b21d5 Change logos, text etc. branding to SickGear. 2014-11-12 20:56:40 +00:00
KontiSR
058f9d1399 This is a check used in webserve.py getManualSearchStatus(), to get the current running queuitem, if it is a manual or failed/retry search. If so, the frontend will report the matching episode as in status "searching".
The function is_manualsearch_in_progress() is introduced for this purpose and only reference in webserve.py.
Previously when a daily search is running, and a manual search is queued behind it, it will wrongly give back that the manual search is running, this could result in unexpected behavior.
2014-09-23 11:06:02 +02:00
echel0n
b0d550b3fb Fix for no attr success 2014-09-21 13:42:01 -07:00
echel0n
f15d5cff57 Revert "Fix for updating queue icon when DailySearchQueueItem is running." 2014-09-21 13:37:14 -07:00
KontiSR
69bb4d6373 Fix for updating queue icon when DailySearchQueueItem is running.
Did not had any impact on searching.. only unexpected behavior when updating the spinner or queue img.
2014-09-21 12:18:43 +02:00
adam
5ac3895600 Fixes daily search and speed improvements 2014-09-20 20:34:00 +08:00
KontiSR
a4c8c53c91 Fixed SearchFailedDownload when selecting multiple ep's as failed in displayShow
Searched for other occurrences for the search_queue.FailedQueueItem() call.
Changed segment to [segment]

The failedDownloadQueueItem always expects a single epObj in the segment. But when checking multiple episodes as "Failed" in displayShow, an array is provided. I've changed the queueItem to always iterate over it as an array. And i've made sure that when retry a single episode, it's also send to the queueItem as an array (with a single element).
I've had to adapt the queuing functionality in webserve for this.
2014-09-19 10:55:05 +02:00
KontiSR
bdac98db4b Implemented the queuing functionality also for Failed downloads.
ajaxEpSearch.js now also processes the clicks for Retries. SO merged those together and removed ajaxEpRetry.js.
2014-09-15 15:01:18 +02:00
KontiSR
419e35f300 sbRoot missing in some img url's
Added queued.png image.. for manual searching

Fixed issue where consecutive manual searches where not queued properly
Added first backend processing for retrieving all queued searches. This should fix the frondend blocking when doing manual searches for episodes, because the backend is free sooner. It now only queues the search.
Created a returning ajax call for getting a list of all searches in queue en running on the displayShow page.

For the getManualSearchStatus() function, only use curItem from the ManualSearchQueueItem or FailedQueueItem threads.

Conflicts:
	sickbeard/search_queue.py
2014-09-15 13:31:16 +02:00
echel0n
627debcf88 Building of name cache executed now at start of searches to prevent building cache for show more then once when not needed 2014-09-15 02:46:08 -07:00
echel0n
67bd1a9e98 Fix for daily searches and high cpu usage plus increases search speed 2014-09-15 01:28:11 -07:00
echel0n
22d7230b3f Fix for AttributeError: 'FailedQueueItem' object has no attribute 'ep_obj' 2014-09-14 17:40:59 -07:00
Adam
981d6612b8 Fixing typo in last commit 2014-09-08 17:45:02 +08:00
Adam
f5d3f71c70 Add search queue info to ManageSearches page 2014-09-08 11:54:43 +08:00
echel0n
3efa9709af Fixed DailySearchQueueItem show attribute error during searches 2014-09-06 22:01:24 -07:00
echel0n
59675f27ac Fixed failed download handling.
Improved search queue code.
2014-09-06 21:36:23 -07:00
echel0n
ee458bd211 Fixed AttributeError: 'NoneType' object has no attribute 'is_scene' in scene_numbering code.
Show root dirs can not be set from general config menu.
Mass editing shows now has the ability to delete root dirs as well as edit them.
Daily search no longer is restricted to just 1 week of results for searching from which now allows for replacing lower quality downloads with higher quality ones if available.
RSS Cache is updated for each provider on demand now when performing manual, failed, backlog, or daily searches.
2014-08-30 01:47:00 -07:00
echel0n
14c354b551 Added proxy support to Indexer APIs.
Provider getURL and downloadResult functions now removed and replaced with ones from helpers.py to help slim the code down plus allow more better control over request sessions.

Removed TVTumbler code.

Fixed HDBits provider.

Fixed config settings that were ment to be booleans but instead where set as str or int, should help resolve random html errors.

XEM Refresh check re-coded.

NameParser code for creating show object has been changed to only attempt at the very end once its found the bestMatch result, helps on resources and performance.
2014-07-27 03:59:21 -07:00
echel0n
c423d349e8 Fixed failed and manual snatches that where causing WebUI lock-ups. 2014-07-19 18:06:04 -07:00
echel0n
c25da850ab Updated scene exception code for checking when last refreshed. 2014-07-19 04:52:55 -07:00
echel0n
6957bd0656 Possible fix for stacked provider names during backlog search.
Possible fix for anidb connection stop issues during restart/shutdown.
2014-07-17 23:16:18 -07:00
echel0n
5bc775dfb9 Fix for threading issues with backlogs and whitelist/blacklist issues for anime shows. 2014-07-17 22:57:35 -07:00
echel0n
70c2a2d130 Fixed errors in scene exceptions when retrieving list of exceptions.
Fixed trakt.tv errors when updating/syncing show lists.
2014-07-17 18:06:42 -07:00
echel0n
d02c0bd6eb Fixed issues with editing/saving custom scene exceptions.
Fixed charmap issues for anime show names.

Fixed issues with display show page and epCat key errors.

Fixed duplicate log messages for clearing provider caches.

Fixed issues with email notifier ep names not properly being encoded to UTF-8.

TVDB<->TVRAGE Indexer ID mapping is now performed on demand to be used when needed such as newznab providers can be searched with tvrage_id's and some will return tvrage_id's that later can be used to create show objects from for faster and more accurate name parsing, mapping is done via Trakt API calls.

Added stop event signals to schedualed tasks, SR now waits indefinate till task has been fully stopped before completing a restart or shutdown event.

NameParserCache is now persistent and stores 200 parsed results at any given time for quicker lookups and better performance, this helps maintain results between updates or shutdown/startup events.

Black and White lists for anime now only get used for anime shows as intended, performance gain for non-anime shows that dont need to load these lists.

Internal name cache now builds it self on demand when needed per show request plus checks if show is already in cache and if true exits routine to save time.

Schedualer and QueueItems classes are now a sub-class of threading.Thread and a stop threading event signal has been added to each.

If I forgot to list something it doesn't mean its not fixed so please test and report back if anything is wrong or has been corrected by this new release.
2014-07-14 19:00:53 -07:00
echel0n
c8d899ad66 Removed maintance schedualer and moved the routines from it to happen before a search is started to ensure things are up to date and to stop waking up synology devices, regexes also made less greedy. 2014-07-07 14:30:27 -07:00
echel0n
a1cef0b958 Added in context to aid in garbage cleanup to help reduce memory usage. 2014-06-30 10:48:18 -07:00
echel0n
64b857ee57 Fixed failed download handling. 2014-06-06 17:06:29 -07:00
echel0n
34da3f53d8 Added Black and White list for release groups for anime shows.
Fixed anime show list splitter.

Added check for anime shows and providers so we don't perform searches with a provider that doesn't support the type of show genre we are searching.
2014-05-27 03:09:57 -07:00
echel0n
c5f933e4c8 Fixed app performance issues from recent upgrades.
Fixed issues with manual searches not working.

Moved misc routine tasks to a proper schedualed thread that runs every 1 hour for updating network timezones, scene exceptions, and trimming failed history.

Forced misc routine tasks at startup to complete first before anything else runs, critical since this does our scene exception updates and internal scene cache loading which is needed before searches begin.
2014-05-26 13:16:07 -07:00