Commit graph

146 commits

Author SHA1 Message Date
JackDandy
208ddfb42a Change prefer modern html5lib over old to prevent display show issue on systems that fail to clean libs.
In rare cases, systems *fail* to remove the deprecated "_base.pyc" file (and probably others) in \lib\html5lib\treebuilders\. Therefore, the startup cleanup process will now list files that cannot be auto deleted - user must then manually delete files listed in "__README-DANGER.txt".
Change add un/pw for cookie support to improve SpeedCD torrent provider.
Change improve handling faults when downloading .torrent files.
Remove TorrentBytes provider.
Change remove redundant log messages for releases never to be cached removing <30% log spam
Change remove redundant log messages for items not found in cache removing <10% log spam
Pep8.
2018-04-26 01:46:53 +01:00
JackDandy
e41b4a442d Fix issue creating xml metadata files.
Change improve searching torrent providers AReign, EZTV, HDB, SkyT, and SCD.
2018-04-16 03:21:22 +01:00
JackDandy
0ead7771de Change improve the manage searches error stats UI and backend functions.
Add the improved error handling to torrent providers.
Change cache_db to always include db upgrade statements when running SG for the first time.
Change split newznab limit logic into a reusable function, hit_limit_update() and use in BTN prov.
Change tweak CSS to make things a little neater with button spacings.
Add the actual time when current limit will expire to the UI.
Change instead of terminology "errors", use "failures".
Change improve BTN error handling.
Change ensure provider name is output to log at times where it wasn't.
Change ensure failed request URLs and POST params are output to log.
Add time of last failure + type to should_skip log message.
Change code DRY and simplification for improved readability.
Change occurrences of "error" to "fail" for consistency.
Add tmr limit handling to omg and change log_failure_url level to warning.
Change log the failed URL when an API hit limit is reached.
Change "hit" to the more universally generic "tmr" - Too Many Requests.
Change Db columns containing "hit_" are renamed "tmr_"
2018-01-26 01:35:02 +00:00
Prinz23
406c9ad6c6 Change reduce number of DB calls for extra_info_no_name.
Add parse repack, proper level to recent search flow.
2017-10-31 20:35:57 +00:00
Prinz23
05758bb75e Change overhaul handling of PROPERS/REPACKS/REAL.
Add restriction to allow only same release group for repacks.
Change try all episode names with 'real', 'repack', 'proper'.
Add flags table to store when one-off operations have been performed.
Change update history.tbl with action SNATCHED_PROPER on existing items.
Change save proper snatches as action SNATCHED_PROPER + quality into history.tbl
Add optional regex lib with fuzzy (errors ~20% of alpanum chars) for extra_info_no_name.
Add tip to search settings/media search about improved matching with optional regex library.
2017-10-30 22:21:55 +00:00
Prinz23
33a36432d8 Change improve newnab autoselect categories.
Change add nzb.org BoxSD and BoxHD categories.
Change improve category exclude logic for newznab.
Change add WEB-DL category to autoselect categories for newznab.
Change comment out HEVC category parsing since it's not auto selected.
2017-08-19 15:53:59 +01:00
Prinz23
3e9a9095a6 Change make indexer lookup optional in NameParser, and deactivate during searches. 2017-07-21 22:39:08 +01:00
Prinz23
d3a7f0ff5e Add smart logic to reduce api hits to newznab server types and improve how nzbs are downloaded.
Add newznab smart logic to avoid missing releases when there are a great many recent releases.
Change improve performance by using newznab server advertised capabilities.
Change config/providers newznab to display only non-default categories.
Change use scene season for wanted segment in backlog if show is scene numbering.
Change combine Manage Searches / Backlog Search / Limited and Full to Force.
Change consolidate limited and full backlog.
Change config / Search / Backlog search frequency to instead spread backlog searches over a number of days.
Change migrate minimum used value for search frequency into new minimum 7 for search spread.
Change restrict nzb providers to 1 backlog batch run per day.
Add to Config/Search/Unaired episodes/Allow episodes that are released early.
Add to Config/Search/Unaired episodes/Use specific api requests to search for early episode releases.
Add use related ids for newznab searches to increase search efficiency.
Add periodic update of related show ids.
Change terminology Edit Show/"Post processing" tab name to "Other".
Add advanced feature "Related show IDs" to Edit Show/Other used for finding episodes and TV info.
Add search info source image links to those that have zero id under Edit Show/Other/"Related show IDs".
Add "set master" button to Edit Show/Other/"Related show IDs" for info source that can be changed.
Change terminology displayShow "Indexers" to "Links" to cover internal and web links.
Change add related show info sources on displayShow page.
Change don't display "temporarily" defunct TVRage image link on displayShow pages unless it is master info source.
Change if a defunct info source is the master of a show then present a link on displayShow to edit related show IDs.
Change simplify the next backlog search run time display in the page footer.
Change try ssl when fetching data thetvdb, imdb, trakt, scene exception.
Change improve reliability to Trakt notifier by using show related id support.
Change improve config/providers newznab categories layout.
Change show loaded log message at start up and include info source.
Change if episode has no airdate then set status to unaired (was skipped).

Technical
Change move scene_exceptions table from cache.db to sickbeard.db.
Add related ids to show obj.
Add use of mapped indexer ids for newznab.
Add indexer to sql in wanted_eps.
Add aired in (scene) season for wanted episodes.
Add need_anime, need_sports, need_sd, need_hd, need_uhd to wanted episodes and added as parameter to update_providers.
Add fix for lib lockfile/mkdirlockfile.
Add set master TV info source logic.
Change harden ui input validation.
Add per action dialog confirmation.
Change to reload page under more events.
Change implement "Mark all added episodes Wanted to search for releases" when setting new info source.
2016-09-21 17:50:27 +01:00
JackDandy
eac69963f0 Change validate and improve specific Torrent provider connections, IPT, KAT, SCC, TPB, TB, TD, TT.
Change refactor cache for torrent providers to reduce code.
Change improve search category selection BMTV, FSH, FF, TB.
Change identify more SD release qualities.
Change update SpeedCD, MoreThan, TVChaosuk.
Add torrent provider HD4Free.
Remove torrent provider BitSoup.
Change only create threads for providers needing a recent search instead of for all enabled.
Add 4489 as experimental value to "Recent search frequency" to use provider freqs instead of fixed width for all.
Fix searching nzb season packs.
Change remove some logging cruft.
2016-06-08 01:55:13 +01:00
JackDandy
4edd26675c Fix alternative unicode show names from breaking search. 2016-02-26 01:07:39 +00:00
JackDandy
43778d8edd Change providers, add some, remove one, fix a snatch issue, rework ignore/require words, refactor code.
Add BitMeTV torrent provider.
Add TVChaosUK torrent provider.
Add HD-Space torrent provider.
Add Shazbat torrent provider.
Remove Animenzb provider.
Change use tvdbid for searching usenet providers.
Change consolidate global and per show ignore and require words functions.
Change "Require word" title and notes on Config Search page to properly describe its functional logic.
Add "Reject Blu-ray M2TS releases" to BTN provider.
Add regular expression capability to ignore and require words by starting wordlist with "regex:".
Add list shows with custom ignore and require words under the global counterparts on the Search Settings page.
Fix failure to search for more than one selected wanted episode.
2015-12-02 01:31:50 +00:00
JackDandy
a574012c38 Change increase the scope and number of non release group text that is identified and removed. 2015-10-16 14:03:19 +01:00
JackDandy
2051268891 Change refactor, PEP8, code convention cleanse for nzb/torrent code and remove newzbin remnants. 2015-07-15 19:19:39 +01:00
JackDandy
afd590d817 Change provider RSS torrent code to use General Config/Advanced/Proxy host setting, simplify and PEP8. 2015-07-03 17:06:55 +01:00
JackDandy
f782567fd1 Change provider SCC login process to use General Config/Advanced/Proxy host setting.
Change refactor SCC to use torrent provider simplification and PEP8.
2015-07-03 14:24:09 +01:00
Adam
37496189f1 Change py2 exception clauses to py2/3 compatible clauses
Add py2/3 regression testing for exception clauses

Any new code added with old py2 style exceptions will now fail the travis test
2015-06-08 22:56:45 +08:00
Adam
cf0ee24cb4 Remove feedcache implementation and library
This functionality really isn't needed and the code offers no benefit for our needs
2015-06-06 15:16:18 +08:00
JackDandy
1242481803 Change typo. 2015-06-04 23:09:36 +01:00
Prinz23
8f69adaada Fix clearing provider cache 2015-04-25 16:48:15 +08:00
Adam
24236eba6f Change provider cache storage structure
Add handling for failed cache database upgrades
2015-04-01 11:25:24 +08:00
adam
7f5651bb41 Fix slow database operations (port from midgetspy/sickbeard) 2015-01-11 12:27:57 +08:00
JackDandy
c16903bd73 Change "Daily" search to "Recent" search. 2015-01-07 19:54:18 +08:00
adam
2af732c731 Fix duplicate entries in cache database 2014-12-14 12:12:00 +00:00
Supremicus
4c8c6b21d5 Change logos, text etc. branding to SickGear. 2014-11-12 20:56:40 +00:00
adam
c26eb39695 Fix for custom newznab providers with leading integer in name 2014-10-10 15:39:13 +08:00
adam
5ac3895600 Fixes daily search and speed improvements 2014-09-20 20:34:00 +08:00
echel0n
59675f27ac Fixed failed download handling.
Improved search queue code.
2014-09-06 21:36:23 -07:00
echel0n
65eda93b2b Fixed high CPU usage during searches, adding conditional check to prevent un-needed name parsing of search results when search result was previously already parsed and checked during filtering of bad releases 2014-09-06 17:25:07 -07:00
echel0n
45fc163434 PEP8 changes for rss feed cache code 2014-08-31 12:56:45 -07:00
echel0n
ee458bd211 Fixed AttributeError: 'NoneType' object has no attribute 'is_scene' in scene_numbering code.
Show root dirs can not be set from general config menu.
Mass editing shows now has the ability to delete root dirs as well as edit them.
Daily search no longer is restricted to just 1 week of results for searching from which now allows for replacing lower quality downloads with higher quality ones if available.
RSS Cache is updated for each provider on demand now when performing manual, failed, backlog, or daily searches.
2014-08-30 01:47:00 -07:00
echel0n
6c15943363 Possible fix for failed to send torrent errors 2014-08-17 22:52:19 -07:00
Adam
96f918742a Fix for omgwtfnzb skipping: release error 2014-08-15 11:43:11 +08:00
adam
22f3a2e41b Consolidate more provider code
Fix tvtorrents issue

Remove old providers
2014-08-13 05:52:59 +08:00
Adam
4b3a680bb3 Tidy provider code 2014-08-06 01:38:16 +08:00
echel0n
efe115f909 Fixed startup issue due to import module issues. 2014-07-30 21:47:17 -07:00
echel0n
9761c6c6a3 Fixed issues with torrent blackhole download issues. 2014-07-30 21:31:54 -07:00
echel0n
e6c8982d2b Testing fix for torrent download issues from ezrss 2014-07-28 21:42:53 -07:00
echel0n
19a89d453c We now check if a torrent url links to a valid file before adding as a verified result to get snatched, this helps prevent issues when attempting to add torrent to client later on to find the url returned nothing resulting in a error. 2014-07-28 12:19:41 -07:00
Adam
fd9be53fcb Anime propers 2014-07-27 21:04:17 +08:00
echel0n
14c354b551 Added proxy support to Indexer APIs.
Provider getURL and downloadResult functions now removed and replaced with ones from helpers.py to help slim the code down plus allow more better control over request sessions.

Removed TVTumbler code.

Fixed HDBits provider.

Fixed config settings that were ment to be booleans but instead where set as str or int, should help resolve random html errors.

XEM Refresh check re-coded.

NameParser code for creating show object has been changed to only attempt at the very end once its found the bestMatch result, helps on resources and performance.
2014-07-27 03:59:21 -07:00
echel0n
f47734446d Fixed further json issues with speedcd provider.
Fixed several providers and ssl cert verify issues.

Air-by-date shows now automatically get there respective season/episode numbers during release parsing.
2014-07-24 11:16:59 -07:00
echel0n
3a2b67330c During backlog/manual/failed searches we now cache disgarded/ignored results pre-parsed for usage later on incase we end up setting a episode to wanted that matches said results, allows for maximum performance and helps limit the waste of resources used. 2014-07-21 23:00:58 -07:00
echel0n
f340f9b073 Fix for sports shows. 2014-07-21 08:49:20 -07:00
echel0n
5ac99b8c5c Fix for extensions being stripped off by mistake made when adding in -RP fix from few commits ago. 2014-07-21 06:29:07 -07:00
echel0n
bd20f9d2ae Merge pull request #647 from adam111316/log_clear_cache
Updating provider clear cache log message
2014-07-19 21:28:44 -07:00
echel0n
423c09fd9e Provider DB connections remain open instead of constantly open/close for misc db calls. 2014-07-19 14:16:05 -07:00
echel0n
d02c0bd6eb Fixed issues with editing/saving custom scene exceptions.
Fixed charmap issues for anime show names.

Fixed issues with display show page and epCat key errors.

Fixed duplicate log messages for clearing provider caches.

Fixed issues with email notifier ep names not properly being encoded to UTF-8.

TVDB<->TVRAGE Indexer ID mapping is now performed on demand to be used when needed such as newznab providers can be searched with tvrage_id's and some will return tvrage_id's that later can be used to create show objects from for faster and more accurate name parsing, mapping is done via Trakt API calls.

Added stop event signals to schedualed tasks, SR now waits indefinate till task has been fully stopped before completing a restart or shutdown event.

NameParserCache is now persistent and stores 200 parsed results at any given time for quicker lookups and better performance, this helps maintain results between updates or shutdown/startup events.

Black and White lists for anime now only get used for anime shows as intended, performance gain for non-anime shows that dont need to load these lists.

Internal name cache now builds it self on demand when needed per show request plus checks if show is already in cache and if true exits routine to save time.

Schedualer and QueueItems classes are now a sub-class of threading.Thread and a stop threading event signal has been added to each.

If I forgot to list something it doesn't mean its not fixed so please test and report back if anything is wrong or has been corrected by this new release.
2014-07-14 19:00:53 -07:00
Adam
ec860cfc1f Updating provider clear cache log message 2014-07-15 01:12:36 +08:00
echel0n
cc07bdb12f Disabled purging of feed-cache items for providers. 2014-07-11 04:43:28 -07:00
echel0n
933fad20c7 Unable to parse filename errors are now set to show via DEBUG logs only to prevent unicode chars from causing misc beeping sounds ... 2014-07-10 23:44:36 -07:00