Commit graph

139 commits

Author SHA1 Message Date
Prinz23
0da750aa4d Detect newznab server type so that Spotweb is listed as (sw nzb) with a specific default image. 2018-06-02 15:43:41 +01:00
JackDandy
2f24a3c8e4 Merge branch 'master' into develop 2018-05-26 19:01:51 +01:00
Prinz23
0644ec2e61 Change add blacklog search terms for anime PROPERS. 2018-05-26 17:16:40 +01:00
JackDandy
aab67a45f7 Change consolidate provider filters into 'Only allow releases that are'.
Add provider filters, Only allow releases that are ...
  'scene releases (srrDB/predb listed)', 'or contain' text or regex,
  'non scene if no recent search results', 'non scene if no active search results',
  'not scene nuked', and 'nuked if no active search results'.

Add scene release checking to PROPER search task.
Change refactor core Proper functions.
Pep8 common tests.
2018-05-17 04:18:44 +01:00
JackDandy
e35375f5ef Change search only once per cycle for shows with multiple episodes that air on the same day. 2018-05-14 01:16:50 +01:00
JackDandy
a6f7ab47e2 Merge branch 'master' into develop 2018-04-26 15:32:07 +01:00
JackDandy
208ddfb42a Change prefer modern html5lib over old to prevent display show issue on systems that fail to clean libs.
In rare cases, systems *fail* to remove the deprecated "_base.pyc" file (and probably others) in \lib\html5lib\treebuilders\. Therefore, the startup cleanup process will now list files that cannot be auto deleted - user must then manually delete files listed in "__README-DANGER.txt".
Change add un/pw for cookie support to improve SpeedCD torrent provider.
Change improve handling faults when downloading .torrent files.
Remove TorrentBytes provider.
Change remove redundant log messages for releases never to be cached removing <30% log spam
Change remove redundant log messages for items not found in cache removing <10% log spam
Pep8.
2018-04-26 01:46:53 +01:00
Prinz23
c1d3487a21 Change increase namecache size and fix deleting items from it when at capacity.
Add thread lock to NameParserCache.
Add exception handling for add to cache.
Change increase NameParserCache to 1000 entries.
Change NameParserCache to OrderedDefaultdict.
Add move_to_end to OrderedDefaultdict class (backported from python 3).
Add first_key, last_key to OrderedDefaultdict.
Change use much faster first_key in NameParserCache.
Change remove import of python pre 2.7 OrderedDict.
2018-04-17 23:59:20 +01:00
JackDandy
12363d01d5 Change various providers.
Change accept theTVDB Url in addshow search field.
Change Nzb.org usenet provider add config scene only/nuked.
Change SpeedCD torrent provider improve copy/paste cookie support.
Change BTScene, LimeTorrents, SkyTorrents, Torlock, Torrentz, TPB torrent providers.
Add AlphaReign, EZTV torrent providers.
2018-04-13 12:11:21 +01:00
Prinz23
8df50c6cdf Change remove some non-release group stuff from newnab results. 2018-02-02 04:25:37 +00:00
JackDandy
0ead7771de Change improve the manage searches error stats UI and backend functions.
Add the improved error handling to torrent providers.
Change cache_db to always include db upgrade statements when running SG for the first time.
Change split newznab limit logic into a reusable function, hit_limit_update() and use in BTN prov.
Change tweak CSS to make things a little neater with button spacings.
Add the actual time when current limit will expire to the UI.
Change instead of terminology "errors", use "failures".
Change improve BTN error handling.
Change ensure provider name is output to log at times where it wasn't.
Change ensure failed request URLs and POST params are output to log.
Add time of last failure + type to should_skip log message.
Change code DRY and simplification for improved readability.
Change occurrences of "error" to "fail" for consistency.
Add tmr limit handling to omg and change log_failure_url level to warning.
Change log the failed URL when an API hit limit is reached.
Change "hit" to the more universally generic "tmr" - Too Many Requests.
Change Db columns containing "hit_" are renamed "tmr_"
2018-01-26 01:35:02 +00:00
Prinz23
f9cc6ed330 Add provider error table to page Manage/Media Search.
Add failure handling, skip provider for x hour(s) depending on count of failures.
Add detection of api hit limit reached.
Add failure count to omgwtfnzbs.
Change improve categories selection (manual search for example).
2018-01-25 20:48:24 +00:00
JackDandy
4dea2ad022 Add config/general/web interface/send security headers (default enabled).
Change use bare minimum requests for all usenet_crawler search modes.
Change use RSS endpoint for usenet_crawler in cache mode.
Fix ensure remaining unicode shownames are correctly 'UTF-8' and url encoded.
Fix omgwtf test of invalid auth, issue when enabling propers, and updating cache.
Change refactor shownames list into reusable show_name_helper methods.
Add season specific naming exceptions to nzb + btn.
2018-01-25 19:49:28 +00:00
Prinz23
11dfd66efd Fix ensure url unicode shownames are correctly 'UTF-8' and urlencoded.
Refactor shownames list into generic provider method, reusable in multiple providers.
Add remove English at the end of release names (drunkenslug).
2018-01-25 16:14:39 +00:00
Prinz23
6da32a5ed0 Add log message for not found on indexer when adding a new show.
Fix upgrade once ARCHIVED setting by postProcessor.
Fix determination of is_first_best_match.
Change improve smart selection of categories in manual and failed search modes.
Change refactor wantedQuality into own function that can be used in multiple places.
Change improve error resistance in neededQualities class.
Add log warning message if wantedQuality or eps_aired_in_season is missing for search.
Add check backlogitem for wantedQuality and add if missing.
Add use wantedQuality list in wantEpisode.
Change don't use wantedQualities for multipart.
2017-12-27 03:14:20 +00:00
JackDandy
ad63608cad Change improve multi episode release search. 2017-12-13 01:41:34 +00:00
Prinz23
05758bb75e Change overhaul handling of PROPERS/REPACKS/REAL.
Add restriction to allow only same release group for repacks.
Change try all episode names with 'real', 'repack', 'proper'.
Add flags table to store when one-off operations have been performed.
Change update history.tbl with action SNATCHED_PROPER on existing items.
Change save proper snatches as action SNATCHED_PROPER + quality into history.tbl
Add optional regex lib with fuzzy (errors ~20% of alpanum chars) for extra_info_no_name.
Add tip to search settings/media search about improved matching with optional regex library.
2017-10-30 22:21:55 +00:00
JackDandy
253f2237cc Merge branch 'master' into develop 2017-08-26 18:20:00 +01:00
Prinz23
11d283910e Change prevent indexer specific release name parts from fudging search logic. 2017-08-26 18:15:06 +01:00
Prinz23
33a36432d8 Change improve newnab autoselect categories.
Change add nzb.org BoxSD and BoxHD categories.
Change improve category exclude logic for newznab.
Change add WEB-DL category to autoselect categories for newznab.
Change comment out HEVC category parsing since it's not auto selected.
2017-08-19 15:53:59 +01:00
Prinz23
09f511c710 Change only use newznab Api key if needed. 2017-07-12 11:56:48 +01:00
JackDandy
bfb13dfa35 Change NewznabProvider specifics on develop arising from hotfix 0.12.9. 2017-02-24 20:08:37 +00:00
JackDandy
231198ab72 Merge branch 'master' into develop 2017-02-24 19:57:28 +00:00
Prinz23
85ef0a6787 Fix issue saving custom NewznabProviders. 2017-02-24 19:43:45 +00:00
Prinz23
87e608f277 Add scheduled backlog switch. 2017-02-15 00:51:52 +00:00
Prinz23
02e24c6f8f Change recent search to use centralised title and URL parser for newznab. 2016-12-28 03:05:21 +00:00
Prinz23
2b27e3bb74 Change don't fetch caps for disabled usenet providers. 2016-12-21 19:42:00 +00:00
Prinz23
e071542b51 Change don't exclude "unknown" from search.
Fix UHD category select in Recent Search.
Change speed improvement in finding needed categories/qualities (sd, hd, uhd).
2016-11-14 01:50:25 +00:00
JackDandy
9d8f397d41 Pep8 and tweaks to newznab logging. 2016-10-25 00:54:34 +01:00
Prinz23
48032005fd Change image cache processing for browse Trakt page.
Change logging for newznab response code 910 and add check for empty response data.
2016-10-24 23:39:04 +01:00
Prinz23
d3a7f0ff5e Add smart logic to reduce api hits to newznab server types and improve how nzbs are downloaded.
Add newznab smart logic to avoid missing releases when there are a great many recent releases.
Change improve performance by using newznab server advertised capabilities.
Change config/providers newznab to display only non-default categories.
Change use scene season for wanted segment in backlog if show is scene numbering.
Change combine Manage Searches / Backlog Search / Limited and Full to Force.
Change consolidate limited and full backlog.
Change config / Search / Backlog search frequency to instead spread backlog searches over a number of days.
Change migrate minimum used value for search frequency into new minimum 7 for search spread.
Change restrict nzb providers to 1 backlog batch run per day.
Add to Config/Search/Unaired episodes/Allow episodes that are released early.
Add to Config/Search/Unaired episodes/Use specific api requests to search for early episode releases.
Add use related ids for newznab searches to increase search efficiency.
Add periodic update of related show ids.
Change terminology Edit Show/"Post processing" tab name to "Other".
Add advanced feature "Related show IDs" to Edit Show/Other used for finding episodes and TV info.
Add search info source image links to those that have zero id under Edit Show/Other/"Related show IDs".
Add "set master" button to Edit Show/Other/"Related show IDs" for info source that can be changed.
Change terminology displayShow "Indexers" to "Links" to cover internal and web links.
Change add related show info sources on displayShow page.
Change don't display "temporarily" defunct TVRage image link on displayShow pages unless it is master info source.
Change if a defunct info source is the master of a show then present a link on displayShow to edit related show IDs.
Change simplify the next backlog search run time display in the page footer.
Change try ssl when fetching data thetvdb, imdb, trakt, scene exception.
Change improve reliability to Trakt notifier by using show related id support.
Change improve config/providers newznab categories layout.
Change show loaded log message at start up and include info source.
Change if episode has no airdate then set status to unaired (was skipped).

Technical
Change move scene_exceptions table from cache.db to sickbeard.db.
Add related ids to show obj.
Add use of mapped indexer ids for newznab.
Add indexer to sql in wanted_eps.
Add aired in (scene) season for wanted episodes.
Add need_anime, need_sports, need_sd, need_hd, need_uhd to wanted episodes and added as parameter to update_providers.
Add fix for lib lockfile/mkdirlockfile.
Add set master TV info source logic.
Change harden ui input validation.
Add per action dialog confirmation.
Change to reload page under more events.
Change implement "Mark all added episodes Wanted to search for releases" when setting new info source.
2016-09-21 17:50:27 +01:00
JackDandy
0d50a4b345 Add indicator for public access search providers.
Change improve probability selecting most seeded release.
Change add the TorrentDay x265 category to search.
Change torrent provider code PEP8 and refactoring.
Add BTScene torrent provider.
Add Extratorrent provider.
Add Limetorrents provider.
Add nCore torrent provider.
Remove Usenet Crawler provider.
2016-09-02 00:29:02 +01:00
Prinz23
a54b89e8ed Add handling for 'part' numbered new releases and also for specific existing 'part' numbered releases. 2016-08-21 21:31:18 +01:00
JackDandy
decbf0f990 Change increase delay between requests to nnab servers to over 2 seconds.
Change add three IPTorrents fallback urls.
Change remove one dead and add three fallback magnet torcaches for blackhole use.
2016-07-25 18:06:21 +01:00
JackDandy
979da54cbf Change nzbs.org anime search category and fix newznab anime backlog search
Change improve nzbgeek search response
Change use query search at 6box (id search fails)
2016-07-02 18:12:27 +01:00
JackDandy
eac69963f0 Change validate and improve specific Torrent provider connections, IPT, KAT, SCC, TPB, TB, TD, TT.
Change refactor cache for torrent providers to reduce code.
Change improve search category selection BMTV, FSH, FF, TB.
Change identify more SD release qualities.
Change update SpeedCD, MoreThan, TVChaosuk.
Add torrent provider HD4Free.
Remove torrent provider BitSoup.
Change only create threads for providers needing a recent search instead of for all enabled.
Add 4489 as experimental value to "Recent search frequency" to use provider freqs instead of fixed width for all.
Fix searching nzb season packs.
Change remove some logging cruft.
2016-06-08 01:55:13 +01:00
JackDandy
4edd26675c Fix alternative unicode show names from breaking search. 2016-02-26 01:07:39 +00:00
JackDandy
1811f7176e Change how starified keys are handled with newznab categories.
Add more logging to newznab modes.
2015-12-14 03:38:17 +00:00
JackDandy
43778d8edd Change providers, add some, remove one, fix a snatch issue, rework ignore/require words, refactor code.
Add BitMeTV torrent provider.
Add TVChaosUK torrent provider.
Add HD-Space torrent provider.
Add Shazbat torrent provider.
Remove Animenzb provider.
Change use tvdbid for searching usenet providers.
Change consolidate global and per show ignore and require words functions.
Change "Require word" title and notes on Config Search page to properly describe its functional logic.
Add "Reject Blu-ray M2TS releases" to BTN provider.
Add regular expression capability to ignore and require words by starting wordlist with "regex:".
Add list shows with custom ignore and require words under the global counterparts on the Search Settings page.
Fix failure to search for more than one selected wanted episode.
2015-12-02 01:31:50 +00:00
JackDandy
7b3f4356c7 Change reduce aggressive use of scene numbering that was overriding user preference where not needed.
Change set "Scene numbering" checkbox and add text to the label tip in third step of add "New Show" if scene numbers are found for the selected show in the search results of the first step.
Change label text on edit show page to highlight when manual numbering and scene numbers are available.
Fix disabling "Scene numbering" of step three in add "New Show" was ignored when scene episode number mappings exist.
Fix don't use scene episode number mappings everywhere when "Scene numbering" is disabled for a show.
Fix width of legend underlining on the third step used to bring other display elements into alignment.
Change when downloading magnet or nzb files, verify the file in cache dir and then move to blackhole.
Change to consistent use of properties is_anime and is_sports in providers.
Change name_parser pep8 and code conventions.
2015-08-16 16:42:33 +01:00
JackDandy
2051268891 Change refactor, PEP8, code convention cleanse for nzb/torrent code and remove newzbin remnants. 2015-07-15 19:19:39 +01:00
JackDandy
75da620475 Change provider TB and TD logo icons used on Config/Search Providers. 2015-07-04 10:31:39 +01:00
JackDandy
f6730c6ec7 Change refactor and code simplification for newznab providers. 2015-07-03 17:32:43 +01:00
JackDandy
f782567fd1 Change provider SCC login process to use General Config/Advanced/Proxy host setting.
Change refactor SCC to use torrent provider simplification and PEP8.
2015-07-03 14:24:09 +01:00
JackDandy
2083dfdb30 Change prevent wasted API hit where show and exception names create a duplicate sanitised year. 2015-04-29 02:15:45 +01:00
Adam
a9da4c2486 Merge branch 'master' into develop 2015-04-25 16:49:50 +08:00
Prinz23
8f69adaada Fix clearing provider cache 2015-04-25 16:48:15 +08:00
Adam
7caef44bb9 Change to consolidate and tidy some provider code [part 2] 2015-04-18 13:17:47 +08:00
Adam
7477a79cb8 Change to consolidate and tidy some provider code 2015-04-18 10:28:44 +08:00
JackDandy
d479fbfc5a Change improve backlog search. 2015-03-29 03:27:59 +01:00