Commit graph

224 commits

Author SHA1 Message Date
JackDandy
50c023a887 Add if all torrent caches fail, save magnets as files for clients (or plugins) that now support it.
Add advice to logs if all caches fail to switch to direct client connect instead of the basic blackhole method.
2016-10-28 13:43:36 +01:00
JackDandy
9d8f397d41 Pep8 and tweaks to newznab logging. 2016-10-25 00:54:34 +01:00
JackDandy
d626c65a24 Change improve parser tolerance for torrent providers.
Change disable TorrentBytes provider, over 90s for a response is not good.
2016-10-02 19:50:24 +01:00
JackDandy
acef8c1ebb Add HD-Torrents provider.
Add freeleech options to fano, freshon, hdspace, phd, ptf providers.
Change SceneTime to cookie auth.
Remove ILT torrent provider.
2016-09-30 23:20:28 +01:00
JackDandy
145301e275 Change only show unaired episodes on Manage/Backlog Overview and Manage/Episode Status Management where relevant.
Change display show page, can mark unaired items with a date to "Wanted" to trigger a manual active backlog search provided search setting "Unaired episodes" is enabled.
2016-09-25 02:18:48 +01:00
JackDandy
9d24ec109f Fix only replace initial quality releases from the upgrade to list. 2016-09-22 13:20:48 +01:00
Prinz23
d3a7f0ff5e Add smart logic to reduce api hits to newznab server types and improve how nzbs are downloaded.
Add newznab smart logic to avoid missing releases when there are a great many recent releases.
Change improve performance by using newznab server advertised capabilities.
Change config/providers newznab to display only non-default categories.
Change use scene season for wanted segment in backlog if show is scene numbering.
Change combine Manage Searches / Backlog Search / Limited and Full to Force.
Change consolidate limited and full backlog.
Change config / Search / Backlog search frequency to instead spread backlog searches over a number of days.
Change migrate minimum used value for search frequency into new minimum 7 for search spread.
Change restrict nzb providers to 1 backlog batch run per day.
Add to Config/Search/Unaired episodes/Allow episodes that are released early.
Add to Config/Search/Unaired episodes/Use specific api requests to search for early episode releases.
Add use related ids for newznab searches to increase search efficiency.
Add periodic update of related show ids.
Change terminology Edit Show/"Post processing" tab name to "Other".
Add advanced feature "Related show IDs" to Edit Show/Other used for finding episodes and TV info.
Add search info source image links to those that have zero id under Edit Show/Other/"Related show IDs".
Add "set master" button to Edit Show/Other/"Related show IDs" for info source that can be changed.
Change terminology displayShow "Indexers" to "Links" to cover internal and web links.
Change add related show info sources on displayShow page.
Change don't display "temporarily" defunct TVRage image link on displayShow pages unless it is master info source.
Change if a defunct info source is the master of a show then present a link on displayShow to edit related show IDs.
Change simplify the next backlog search run time display in the page footer.
Change try ssl when fetching data thetvdb, imdb, trakt, scene exception.
Change improve reliability to Trakt notifier by using show related id support.
Change improve config/providers newznab categories layout.
Change show loaded log message at start up and include info source.
Change if episode has no airdate then set status to unaired (was skipped).

Technical
Change move scene_exceptions table from cache.db to sickbeard.db.
Add related ids to show obj.
Add use of mapped indexer ids for newznab.
Add indexer to sql in wanted_eps.
Add aired in (scene) season for wanted episodes.
Add need_anime, need_sports, need_sd, need_hd, need_uhd to wanted episodes and added as parameter to update_providers.
Add fix for lib lockfile/mkdirlockfile.
Add set master TV info source logic.
Change harden ui input validation.
Add per action dialog confirmation.
Change to reload page under more events.
Change implement "Mark all added episodes Wanted to search for releases" when setting new info source.
2016-09-21 17:50:27 +01:00
JackDandy
05a9dc62f1 Add Torrentz2 provider.
Change handle when a torrent provider goes down and its urls are cleared.
Add handler for when rar files can not be opened during post processing.
2016-09-12 17:07:22 +01:00
JackDandy
0d50a4b345 Add indicator for public access search providers.
Change improve probability selecting most seeded release.
Change add the TorrentDay x265 category to search.
Change torrent provider code PEP8 and refactoring.
Add BTScene torrent provider.
Add Extratorrent provider.
Add Limetorrents provider.
Add nCore torrent provider.
Remove Usenet Crawler provider.
2016-09-02 00:29:02 +01:00
JackDandy
48a27e57a0 Add detection of password protected rars with config/Post Processing/'Unpack downloads' enabled.
Change post process to cleanup filenames with config/Post Processing/'Unpack downloads' enabled.
Change post process to join incrementally named (i.e. file.001 to file.nnn) split files.
Change replace unrar2 lib with rarfile 3.0 and UnRAR.exe 5.40 beta 4 freeware.
Change post process "Copy" to delete redundant files after use.
2016-08-22 16:09:07 +01:00
Prinz23
a54b89e8ed Add handling for 'part' numbered new releases and also for specific existing 'part' numbered releases. 2016-08-21 21:31:18 +01:00
JackDandy
decbf0f990 Change increase delay between requests to nnab servers to over 2 seconds.
Change add three IPTorrents fallback urls.
Change remove one dead and add three fallback magnet torcaches for blackhole use.
2016-07-25 18:06:21 +01:00
JackDandy
c37ed638e4 Add qBitTorrent to Search Settings/Torrent Search.
Add "Test NZBGet" client to Search Settings/NZB Search/NZBGet.
2016-07-04 00:23:11 +01:00
JackDandy
979da54cbf Change nzbs.org anime search category and fix newznab anime backlog search
Change improve nzbgeek search response
Change use query search at 6box (id search fails)
2016-07-02 18:12:27 +01:00
JackDandy
4fb5d6ea90 Change logged in and a fail-over signature. 2016-06-21 02:06:56 +01:00
JackDandy
be9723e719 Add Zooqle torrent provider. 2016-06-13 19:50:44 +01:00
JackDandy
76cc7d4913 Add PrivateHD torrent provider. 2016-06-13 19:43:22 +01:00
JackDandy
eac69963f0 Change validate and improve specific Torrent provider connections, IPT, KAT, SCC, TPB, TB, TD, TT.
Change refactor cache for torrent providers to reduce code.
Change improve search category selection BMTV, FSH, FF, TB.
Change identify more SD release qualities.
Change update SpeedCD, MoreThan, TVChaosuk.
Add torrent provider HD4Free.
Remove torrent provider BitSoup.
Change only create threads for providers needing a recent search instead of for all enabled.
Add 4489 as experimental value to "Recent search frequency" to use provider freqs instead of fixed width for all.
Fix searching nzb season packs.
Change remove some logging cruft.
2016-06-08 01:55:13 +01:00
JackDandy
dd35c7b9f6 Change add optional login URL. 2016-03-21 11:10:24 +00:00
JackDandy
4edd26675c Fix alternative unicode show names from breaking search. 2016-02-26 01:07:39 +00:00
JackDandy
df4e61c661 Add Emby notifier to config/Notifications (thanks to Supremicus for the jumpstart) with a homebrew description. 2016-02-25 12:56:55 +00:00
Prinz23
0bbaefe17b Change disable connection attempts and remove UI references to the TVRage info source.
Change TVDB to be the default indexer if selected indexer is not configured active.
Change add dummyIndexer for inactive Indexers.
Change don't update show for indexer that are marked inactive.
2015-12-22 12:54:41 +00:00
JackDandy
43778d8edd Change providers, add some, remove one, fix a snatch issue, rework ignore/require words, refactor code.
Add BitMeTV torrent provider.
Add TVChaosUK torrent provider.
Add HD-Space torrent provider.
Add Shazbat torrent provider.
Remove Animenzb provider.
Change use tvdbid for searching usenet providers.
Change consolidate global and per show ignore and require words functions.
Change "Require word" title and notes on Config Search page to properly describe its functional logic.
Add "Reject Blu-ray M2TS releases" to BTN provider.
Add regular expression capability to ignore and require words by starting wordlist with "regex:".
Add list shows with custom ignore and require words under the global counterparts on the Search Settings page.
Fix failure to search for more than one selected wanted episode.
2015-12-02 01:31:50 +00:00
JackDandy
0c75259102 Remove Strike torrent provider and add thetorrent cache for public hash use. 2015-10-16 13:44:30 +01:00
JackDandy
ba566bccd7 Add Torrenting provider. 2015-09-17 01:02:37 +01:00
JackDandy
6c76628d5d Change downloads to prevent cache misfiring with "Result is not a valid torrent file". 2015-09-15 22:13:02 +01:00
JackDandy
1f18ae4212 Add seed time per provider for torrent clients that support seed time per torrent, i.e. currently only uTorrent.
Remove seed time display for Transmission in config/Torrent Search page because the torrent client doesn't support it.
2015-09-13 16:46:37 +01:00
JackDandy
7b3f4356c7 Change reduce aggressive use of scene numbering that was overriding user preference where not needed.
Change set "Scene numbering" checkbox and add text to the label tip in third step of add "New Show" if scene numbers are found for the selected show in the search results of the first step.
Change label text on edit show page to highlight when manual numbering and scene numbers are available.
Fix disabling "Scene numbering" of step three in add "New Show" was ignored when scene episode number mappings exist.
Fix don't use scene episode number mappings everywhere when "Scene numbering" is disabled for a show.
Fix width of legend underlining on the third step used to bring other display elements into alignment.
Change when downloading magnet or nzb files, verify the file in cache dir and then move to blackhole.
Change to consistent use of properties is_anime and is_sports in providers.
Change name_parser pep8 and code conventions.
2015-08-16 16:42:33 +01:00
JackDandy
2051268891 Change refactor, PEP8, code convention cleanse for nzb/torrent code and remove newzbin remnants. 2015-07-15 19:19:39 +01:00
JackDandy
6f1e77139e Add BeyondHD torrent provider. 2015-07-06 12:15:40 +01:00
JackDandy
f782567fd1 Change provider SCC login process to use General Config/Advanced/Proxy host setting.
Change refactor SCC to use torrent provider simplification and PEP8.
2015-07-03 14:24:09 +01:00
JackDandy
47347884cb Fix fault matching air by date shows by using correct episode/season strings in find search results. 2015-06-09 15:43:54 +01:00
JackDandy
93c219d82e Add Rarbg torrent provider.
Note: The merging of all qualities into one category in API v2 mean more results likely to be cropped. The vendor switched to this method from their (imho, better) original of listing category numbers. It was suggested that clients add and maintain all possible search strings that signify as quality. However, this would still trigger misses, introduce other issues, plus add needless non-standard API complexity. Conclusion; as designed, SG will utilise other configured providers when this one fails to hit targets, this provider should not be configured as a sole provider.
2015-06-08 13:04:59 +01:00
JackDandy
56a3411649 Change refactor and code simplification for torrent providers. 2015-06-06 06:31:20 +01:00
Adam
7caef44bb9 Change to consolidate and tidy some provider code [part 2] 2015-04-18 13:17:47 +08:00
Adam
7477a79cb8 Change to consolidate and tidy some provider code 2015-04-18 10:28:44 +08:00
JackDandy
c16903bd73 Change "Daily" search to "Recent" search. 2015-01-07 19:54:18 +08:00
Supremicus
4c8c6b21d5 Change logos, text etc. branding to SickGear. 2014-11-12 20:56:40 +00:00
KontiSR
0d415fcc20 This is a first attempt of getting the season pack downloading to work.
Unfortunately some of the old code I modified is still not 100% clear to me.. so I can't really oversee if this will work. That's why I suggest, this should be extensively tested, before merging into master!

I broke backlog searching.. Should be working again.

Tested, and seems to be working fine!
2014-09-25 13:18:07 +02:00
adam
5ac3895600 Fixes daily search and speed improvements 2014-09-20 20:34:00 +08:00
KontiSR
4c8b100be9 Shouldn't be copy() but update().
Tested now, and at least the exception is gone.
2014-09-17 13:43:10 +02:00
KontiSR
4bfb271e8d Can't += dicts. Need to dict.copy()
Bug in master while doing daily search: https://sickrage.tv/forums/forum/help-support/bug-issue-reports/8358-unsupported-operand-type-s-for-dict-and-dict
2014-09-17 10:18:47 +02:00
echel0n
67bd1a9e98 Fix for daily searches and high cpu usage plus increases search speed 2014-09-15 01:28:11 -07:00
Adam
1790b3be7f Add TokyoToshokanProvider support 2014-09-12 19:54:49 +08:00
echel0n
8f6d014830 Fixed issues with cache results being used during searches 2014-09-07 02:41:21 -07:00
echel0n
2193a4bfd1 Fixed search issues regarding error about result attribute being referenced early 2014-09-07 02:08:24 -07:00
echel0n
20c0b4ea72 Fixed global name season error during searches 2014-09-06 22:25:01 -07:00
echel0n
59675f27ac Fixed failed download handling.
Improved search queue code.
2014-09-06 21:36:23 -07:00
echel0n
4af28657af Fixed TypeError: list indices must be integers, not list that occured during searches 2014-09-06 16:55:39 -07:00
djoole
159e8ceb7c Added the torrent provider for www.t411.me tracker 2014-09-06 02:16:04 +02:00
echel0n
ee458bd211 Fixed AttributeError: 'NoneType' object has no attribute 'is_scene' in scene_numbering code.
Show root dirs can not be set from general config menu.
Mass editing shows now has the ability to delete root dirs as well as edit them.
Daily search no longer is restricted to just 1 week of results for searching from which now allows for replacing lower quality downloads with higher quality ones if available.
RSS Cache is updated for each provider on demand now when performing manual, failed, backlog, or daily searches.
2014-08-30 01:47:00 -07:00
echel0n
bcffc09589 Replaced provider backlog only search option with 2 new options that allow you to enable daily searches and backlog searches individually per provider, default is enabled for both. 2014-08-28 22:16:25 -07:00
Adam
f43c214069 Fixes rejection of invalid torrent files 2014-08-26 12:57:40 +08:00
echel0n
4b6602797f Fix for newznab provider searches unicode issues 2014-08-23 14:40:13 -07:00
echel0n
be17ed122b Fix for air-by-date and sports shows issues with parsing results. 2014-08-22 00:21:37 -07:00
echel0n
8415b32fc6 Possible fix for failed to send torrent errors 2014-08-17 22:51:45 -07:00
adam
22f3a2e41b Consolidate more provider code
Fix tvtorrents issue

Remove old providers
2014-08-13 05:52:59 +08:00
echel0n
9761c6c6a3 Fixed issues with torrent blackhole download issues. 2014-07-30 21:31:54 -07:00
echel0n
e6c8982d2b Testing fix for torrent download issues from ezrss 2014-07-28 21:42:53 -07:00
echel0n
19a89d453c We now check if a torrent url links to a valid file before adding as a verified result to get snatched, this helps prevent issues when attempting to add torrent to client later on to find the url returned nothing resulting in a error. 2014-07-28 12:19:41 -07:00
echel0n
60205979a8 Merge pull request #667 from adam111316/anime_propers_3
Anime proper support
2014-07-27 07:17:55 -07:00
echel0n
9493a8f7e5 Added back in missing code for finding propers, was removed by mistake.
Fixed issue with torrent clients not snatching due to improper handling of torrent hashes/files.
2014-07-27 07:04:37 -07:00
Adam
fd9be53fcb Anime propers 2014-07-27 21:04:17 +08:00
echel0n
14c354b551 Added proxy support to Indexer APIs.
Provider getURL and downloadResult functions now removed and replaced with ones from helpers.py to help slim the code down plus allow more better control over request sessions.

Removed TVTumbler code.

Fixed HDBits provider.

Fixed config settings that were ment to be booleans but instead where set as str or int, should help resolve random html errors.

XEM Refresh check re-coded.

NameParser code for creating show object has been changed to only attempt at the very end once its found the bestMatch result, helps on resources and performance.
2014-07-27 03:59:21 -07:00
echel0n
0e5fbbc5c3 Fix for incorrect show snatches 2014-07-24 21:29:03 -07:00
echel0n
de5db9be64 Added ability to set a default indexer for trakt notifier used for adding shows from watch list so that SickRage knows what indexer to set the show as.
Indexer mapping now uses indexer api calls to gather its information and then stores it to a new table called indexer_mapping for instant lookups later on.

Fixed trakt related issues for adding new shows and syncing.

Centered items at bottom of pages to just look a little nicer and fit properly.
2014-07-23 22:07:01 -07:00
echel0n
3a2b67330c During backlog/manual/failed searches we now cache disgarded/ignored results pre-parsed for usage later on incase we end up setting a episode to wanted that matches said results, allows for maximum performance and helps limit the waste of resources used. 2014-07-21 23:00:58 -07:00
echel0n
899d03b5fe Fixed issue with appending UNNOWN qualities to provider results after being sorted. 2014-07-21 18:48:16 -07:00
echel0n
bf41ba59be Fixed sorting provider results by quality code, was causing a error. 2014-07-21 17:55:45 -07:00
echel0n
f340f9b073 Fix for sports shows. 2014-07-21 08:49:20 -07:00
echel0n
6f817c0c5c Quality sorting fixed for provider results.
Reverted newznab search strings being confined to just season till we can find a better way of correcting the issue with Usenet Crawler.
2014-07-21 08:38:56 -07:00
echel0n
39054d7810 Fix for validating episode data during provider result gathering. 2014-07-21 07:21:03 -07:00
echel0n
5ac99b8c5c Fix for extensions being stripped off by mistake made when adding in -RP fix from few commits ago. 2014-07-21 06:29:07 -07:00
echel0n
57cf86279b Places Quality UNKNOWN items from provider results at end of itemsList leaving order of sorted items highest to lowest. 2014-07-20 23:59:57 -07:00
echel0n
d6442e0685 Provider results get sorted by quality before filtering occures. 2014-07-20 23:43:52 -07:00
echel0n
af0ccd65cf Fixes for a few provider issues plus passing of search_mode variable. 2014-07-20 22:47:13 -07:00
echel0n
ff5107cfe2 Provider searches for backlog, manual, and failed have been re-worked to not hammer the providers so much plus perform alot faster. 2014-07-20 22:07:53 -07:00
echel0n
d02c0bd6eb Fixed issues with editing/saving custom scene exceptions.
Fixed charmap issues for anime show names.

Fixed issues with display show page and epCat key errors.

Fixed duplicate log messages for clearing provider caches.

Fixed issues with email notifier ep names not properly being encoded to UTF-8.

TVDB<->TVRAGE Indexer ID mapping is now performed on demand to be used when needed such as newznab providers can be searched with tvrage_id's and some will return tvrage_id's that later can be used to create show objects from for faster and more accurate name parsing, mapping is done via Trakt API calls.

Added stop event signals to schedualed tasks, SR now waits indefinate till task has been fully stopped before completing a restart or shutdown event.

NameParserCache is now persistent and stores 200 parsed results at any given time for quicker lookups and better performance, this helps maintain results between updates or shutdown/startup events.

Black and White lists for anime now only get used for anime shows as intended, performance gain for non-anime shows that dont need to load these lists.

Internal name cache now builds it self on demand when needed per show request plus checks if show is already in cache and if true exits routine to save time.

Schedualer and QueueItems classes are now a sub-class of threading.Thread and a stop threading event signal has been added to each.

If I forgot to list something it doesn't mean its not fixed so please test and report back if anything is wrong or has been corrected by this new release.
2014-07-14 19:00:53 -07:00
echel0n
0e3495d30f Fix for show parsing errors to be displayed via debug logs now instead as a warning. 2014-07-11 00:59:05 -07:00
echel0n
7a95b0bb14 Added new InvalidShowException to NameParser, Removed isGoodResult search check and now attached parsed result show objects and compare them against the show object we use to search with instead. 2014-07-06 06:11:04 -07:00
echel0n
a3a3777d33 Fixed blackAndWhitelist for anime release groups. 2014-07-06 04:12:16 -07:00
echel0n
f0146f728e Lowered CPU usage even further, re-wrote daemonizer code for startups, removed misc unrequired functions from providers. 2014-07-03 07:07:07 -07:00
echel0n
a1cef0b958 Added in context to aid in garbage cleanup to help reduce memory usage. 2014-06-30 10:48:18 -07:00
echel0n
e0eea79ec5 Fix for ABD and Sports shows issue with toordinal function. 2014-06-25 21:39:34 -07:00
echel0n
7e0bb651b2 Database now closes connection after each query and reconnects if closed to ensure no more connection and locked database problems.
Fixed more internal 500 errors.
2014-06-21 15:46:59 -07:00
echel0n
d00d55fdfc Modified DB code to close its connection if right after its finished with it, helps performance-wise and should resolve locked db issues as well. 2014-06-07 14:32:38 -07:00
echel0n
24dfbc3f15 Quality is now set during parsing of results.
Fixed more anime regex bugs.
Fixed Indexer API issues for anime shows and unicode problems.
2014-06-07 04:06:21 -07:00
echel0n
d7396896b5 Added in regex matching with fuzzy matching.
Shows now display Indexer absolute numbering.
Improved speed of parsing search results.
Fixed episode naming issues.
2014-05-31 03:35:11 -07:00
echel0n
f01c5852d4 Improved find propers code.
Backlog and manual searches now check cache first for search results.
2014-05-30 00:51:24 -07:00
echel0n
e009641804 SickRage now stores the XEM scene data to the main DB under tv_episodes for faster lookups and better linkages with epiosde objects, insures we have scene data on hand when we want it!
Adding functions to lookup scene season and ep info via scene absolute numbers.

We now try and create a show object from parsed release info before we consider it a valid parsed result to insure we even have the show in our show list, also used to validate anime releases.

Misc bug fixes here and there.
2014-05-28 14:16:03 -07:00
echel0n
34da3f53d8 Added Black and White list for release groups for anime shows.
Fixed anime show list splitter.

Added check for anime shows and providers so we don't perform searches with a provider that doesn't support the type of show genre we are searching.
2014-05-27 03:09:57 -07:00
echel0n
f7b11e1e98 Fixed issues with post-processing.
Fixed issue with priority post-processing, bug would allow 2nd attempt to go through even if priority settings were not selected.

Fixed issues with cache retrieval and storing of provider results, properly stored now with season and ep info converted to indexer numbering.

Fixed a bug that was caching unconverted scene numbered files for post-processing causing us to skip converting them when needed.
2014-05-26 03:42:34 -07:00
echel0n
9a3e7ab0a9 Added anime support with anidb support.
Added fanzub anime nzb provider.
Fixed NyaaTorrents anime provider.

This is in testing phase so bugs are to be expected.
2014-05-25 23:29:22 -07:00
echel0n
7391a6e454 App name change in code. 2014-05-23 05:37:22 -07:00
echel0n
fa11b4ef3a Removed some sleep timers to improve overall speed of sickrage. 2014-05-20 11:01:51 -07:00
echel0n
c6b064de23 Fixed small typo causing scene numbering to indexer numbering issues 2014-05-19 06:57:20 -07:00
echel0n
66e499ab23 Fixes issues with daily searcher not snatching its results when it has any to snatch.
Fixes issues with not properly converting scene numbering eps to indexer numbering.
2014-05-19 06:08:16 -07:00
echel0n
de20d13c61 Added per-provider option 'backlog only' which allows you to disclude any provider from the daily search but still use it for backlogs, currently working for torrents only and will add to nzb providers later on today.
Cleaned up details on search options for providers and made it look nicer
2014-05-18 09:39:30 -07:00
echel0n
c350c0fdf9 Fixed dupe issues with backlog searches.
Set cache to store results for 1 week and remove anything older then 1 week only if daily search has searched the results before hand.
2014-05-18 08:33:31 -07:00
echel0n
c65573a8d7 Re-wrote daily searcher to search for unaired and wanted episodes going back as far as 1 week, also moved it so it queue's its items now.
Cleaned up description of new per-provider season search options.
2014-05-18 05:59:42 -07:00