Commit graph

79 commits

Author SHA1 Message Date
JackDandy
0ead7771de Change improve the manage searches error stats UI and backend functions.
Add the improved error handling to torrent providers.
Change cache_db to always include db upgrade statements when running SG for the first time.
Change split newznab limit logic into a reusable function, hit_limit_update() and use in BTN prov.
Change tweak CSS to make things a little neater with button spacings.
Add the actual time when current limit will expire to the UI.
Change instead of terminology "errors", use "failures".
Change improve BTN error handling.
Change ensure provider name is output to log at times where it wasn't.
Change ensure failed request URLs and POST params are output to log.
Add time of last failure + type to should_skip log message.
Change code DRY and simplification for improved readability.
Change occurrences of "error" to "fail" for consistency.
Add tmr limit handling to omg and change log_failure_url level to warning.
Change log the failed URL when an API hit limit is reached.
Change "hit" to the more universally generic "tmr" - Too Many Requests.
Change Db columns containing "hit_" are renamed "tmr_"
2018-01-26 01:35:02 +00:00
JackDandy
4dea2ad022 Add config/general/web interface/send security headers (default enabled).
Change use bare minimum requests for all usenet_crawler search modes.
Change use RSS endpoint for usenet_crawler in cache mode.
Fix ensure remaining unicode shownames are correctly 'UTF-8' and url encoded.
Fix omgwtf test of invalid auth, issue when enabling propers, and updating cache.
Change refactor shownames list into reusable show_name_helper methods.
Add season specific naming exceptions to nzb + btn.
2018-01-25 19:49:28 +00:00
JackDandy
e35b67e06f Change enable TorrentBytes provider and improve handling of relative download links from providers. 2017-10-15 02:05:33 +01:00
Prinz23
33a36432d8 Change improve newnab autoselect categories.
Change add nzb.org BoxSD and BoxHD categories.
Change improve category exclude logic for newznab.
Change add WEB-DL category to autoselect categories for newznab.
Change comment out HEVC category parsing since it's not auto selected.
2017-08-19 15:53:59 +01:00
JackDandy
a0fe23b4e8 Change BTN API hostname. 2017-02-19 15:04:18 +00:00
JackDandy
cf5f6e3891 Change accept lists in JSON responses.
Change do not log error for empty BTN un/pw in most cases.
Change BTN to only try API once when doing alternative name searches.
Change when API fails, warn users as a tip that they can configure un/pw.
2017-02-17 14:58:04 +00:00
JackDandy
d97bb7174d Remove Torrentshack.
Change improve provider title processing.
Change improve handling erroneous JSON responses.
Change improve find show with unicode characters.
Change improve result for providers Omgwtf, SpeedCD, Transmithenet, Zoogle.
Change validate .torrent files that contain optional header data.
Fix case where an episode status was not restored on failure.
Add raise log error if no wanted qualities are found.
Change add un/pw to Config/Media providers/Options for BTN API graceful fallback (can remove Api key for security).
Change only download torrent once when using blackhole.
Add Cloudflare module 1.6.8 (be0a536) to mitigate CloudFlare (IUAM) access validator.
Add Js2Py 0.43 (c1442f1) Cloudflare dependency.
Add pyjsparser 2.4.5 (cd5b829) Js2Py dependency.
2017-02-17 04:48:49 +00:00
JackDandy
0d50a4b345 Add indicator for public access search providers.
Change improve probability selecting most seeded release.
Change add the TorrentDay x265 category to search.
Change torrent provider code PEP8 and refactoring.
Add BTScene torrent provider.
Add Extratorrent provider.
Add Limetorrents provider.
Add nCore torrent provider.
Remove Usenet Crawler provider.
2016-09-02 00:29:02 +01:00
JackDandy
052a1a9c38 Change position of parsed qualities to the start of log lines and add log detail to providers that POST. 2015-12-02 17:03:55 +00:00
JackDandy
43778d8edd Change providers, add some, remove one, fix a snatch issue, rework ignore/require words, refactor code.
Add BitMeTV torrent provider.
Add TVChaosUK torrent provider.
Add HD-Space torrent provider.
Add Shazbat torrent provider.
Remove Animenzb provider.
Change use tvdbid for searching usenet providers.
Change consolidate global and per show ignore and require words functions.
Change "Require word" title and notes on Config Search page to properly describe its functional logic.
Add "Reject Blu-ray M2TS releases" to BTN provider.
Add regular expression capability to ignore and require words by starting wordlist with "regex:".
Add list shows with custom ignore and require words under the global counterparts on the Search Settings page.
Fix failure to search for more than one selected wanted episode.
2015-12-02 01:31:50 +00:00
JackDandy
7b3f4356c7 Change reduce aggressive use of scene numbering that was overriding user preference where not needed.
Change set "Scene numbering" checkbox and add text to the label tip in third step of add "New Show" if scene numbers are found for the selected show in the search results of the first step.
Change label text on edit show page to highlight when manual numbering and scene numbers are available.
Fix disabling "Scene numbering" of step three in add "New Show" was ignored when scene episode number mappings exist.
Fix don't use scene episode number mappings everywhere when "Scene numbering" is disabled for a show.
Fix width of legend underlining on the third step used to bring other display elements into alignment.
Change when downloading magnet or nzb files, verify the file in cache dir and then move to blackhole.
Change to consistent use of properties is_anime and is_sports in providers.
Change name_parser pep8 and code conventions.
2015-08-16 16:42:33 +01:00
JackDandy
2051268891 Change refactor, PEP8, code convention cleanse for nzb/torrent code and remove newzbin remnants. 2015-07-15 19:19:39 +01:00
JackDandy
2be6ba08a7 Change provider BTN code simplify and PEP8. 2015-07-03 17:14:01 +01:00
Adam
37496189f1 Change py2 exception clauses to py2/3 compatible clauses
Add py2/3 regression testing for exception clauses

Any new code added with old py2 style exceptions will now fail the travis test
2015-06-08 22:56:45 +08:00
JackDandy
2083dfdb30 Change prevent wasted API hit where show and exception names create a duplicate sanitised year. 2015-04-29 02:15:45 +01:00
Adam
7caef44bb9 Change to consolidate and tidy some provider code [part 2] 2015-04-18 13:17:47 +08:00
Adam
7477a79cb8 Change to consolidate and tidy some provider code 2015-04-18 10:28:44 +08:00
JackDandy
d479fbfc5a Change improve backlog search. 2015-03-29 03:27:59 +01:00
adam
c02de5eea5 Merge branch 'feature/FixBtnAnime' into develop 2015-01-05 22:50:53 +08:00
adam
3c295485ee Add BTN api call parameter debug logging 2015-01-05 22:47:33 +08:00
Adam
7b969b334b Fix anime searches on BTN provider 2015-01-03 21:09:05 +08:00
Supremicus
4c8c6b21d5 Change logos, text etc. branding to SickGear. 2014-11-12 20:56:40 +00:00
echel0n
ee458bd211 Fixed AttributeError: 'NoneType' object has no attribute 'is_scene' in scene_numbering code.
Show root dirs can not be set from general config menu.
Mass editing shows now has the ability to delete root dirs as well as edit them.
Daily search no longer is restricted to just 1 week of results for searching from which now allows for replacing lower quality downloads with higher quality ones if available.
RSS Cache is updated for each provider on demand now when performing manual, failed, backlog, or daily searches.
2014-08-30 01:47:00 -07:00
adam
22f3a2e41b Consolidate more provider code
Fix tvtorrents issue

Remove old providers
2014-08-13 05:52:59 +08:00
Adam
4b3a680bb3 Tidy provider code 2014-08-06 01:38:16 +08:00
echel0n
14c354b551 Added proxy support to Indexer APIs.
Provider getURL and downloadResult functions now removed and replaced with ones from helpers.py to help slim the code down plus allow more better control over request sessions.

Removed TVTumbler code.

Fixed HDBits provider.

Fixed config settings that were ment to be booleans but instead where set as str or int, should help resolve random html errors.

XEM Refresh check re-coded.

NameParser code for creating show object has been changed to only attempt at the very end once its found the bestMatch result, helps on resources and performance.
2014-07-27 03:59:21 -07:00
echel0n
de5db9be64 Added ability to set a default indexer for trakt notifier used for adding shows from watch list so that SickRage knows what indexer to set the show as.
Indexer mapping now uses indexer api calls to gather its information and then stores it to a new table called indexer_mapping for instant lookups later on.

Fixed trakt related issues for adding new shows and syncing.

Centered items at bottom of pages to just look a little nicer and fit properly.
2014-07-23 22:07:01 -07:00
echel0n
af0ccd65cf Fixes for a few provider issues plus passing of search_mode variable. 2014-07-20 22:47:13 -07:00
echel0n
d02c0bd6eb Fixed issues with editing/saving custom scene exceptions.
Fixed charmap issues for anime show names.

Fixed issues with display show page and epCat key errors.

Fixed duplicate log messages for clearing provider caches.

Fixed issues with email notifier ep names not properly being encoded to UTF-8.

TVDB<->TVRAGE Indexer ID mapping is now performed on demand to be used when needed such as newznab providers can be searched with tvrage_id's and some will return tvrage_id's that later can be used to create show objects from for faster and more accurate name parsing, mapping is done via Trakt API calls.

Added stop event signals to schedualed tasks, SR now waits indefinate till task has been fully stopped before completing a restart or shutdown event.

NameParserCache is now persistent and stores 200 parsed results at any given time for quicker lookups and better performance, this helps maintain results between updates or shutdown/startup events.

Black and White lists for anime now only get used for anime shows as intended, performance gain for non-anime shows that dont need to load these lists.

Internal name cache now builds it self on demand when needed per show request plus checks if show is already in cache and if true exits routine to save time.

Schedualer and QueueItems classes are now a sub-class of threading.Thread and a stop threading event signal has been added to each.

If I forgot to list something it doesn't mean its not fixed so please test and report back if anything is wrong or has been corrected by this new release.
2014-07-14 19:00:53 -07:00
echel0n
56e2c28bad SickRage now uses a internal name cache comprised of (showlist, scene exceptions, scene names) used to search and create show objects from which is needed for parsing search results.
Scene exceptions now uses a internal cache for scene exceptions and scene season exceptions, helps reduce overhead to DB and performs faster lookups when making scene exception requests.
2014-07-03 10:30:15 -07:00
echel0n
f0146f728e Lowered CPU usage even further, re-wrote daemonizer code for startups, removed misc unrequired functions from providers. 2014-07-03 07:07:07 -07:00
echel0n
12ac388dc2 Fixed startup/restart/shutdown issues on Windows, Linux, FreeBSD platforms tested.
Fixed for updating issues.

Fixed high cpu and memory usage.
2014-07-02 11:51:14 -07:00
echel0n
a1cef0b958 Added in context to aid in garbage cleanup to help reduce memory usage. 2014-06-30 10:48:18 -07:00
echel0n
75f69396d4 Reverted some changes. 2014-06-30 08:57:32 -07:00
echel0n
5e507b3849 More memory reductions and cleanups 2014-06-30 04:44:36 -07:00
echel0n
61e1e5d2c2 Fix for __exit__ DB errors 2014-06-21 18:35:57 -07:00
JackDandy
42f018bf06 Port:Fix BTN adding same search parameters multiple times. 2014-06-19 16:36:13 +01:00
echel0n
a1da7df12e Fixes for DB issues 2014-06-07 15:35:16 -07:00
echel0n
05dcdd72d7 Fix for air-by-date/sports/anime provider searches 2014-06-06 16:55:14 -07:00
echel0n
997896a540 Fix for transactionals and None Types in sql lists 2014-05-30 04:42:31 -07:00
echel0n
e009641804 SickRage now stores the XEM scene data to the main DB under tv_episodes for faster lookups and better linkages with epiosde objects, insures we have scene data on hand when we want it!
Adding functions to lookup scene season and ep info via scene absolute numbers.

We now try and create a show object from parsed release info before we consider it a valid parsed result to insure we even have the show in our show list, also used to validate anime releases.

Misc bug fixes here and there.
2014-05-28 14:16:03 -07:00
echel0n
34da3f53d8 Added Black and White list for release groups for anime shows.
Fixed anime show list splitter.

Added check for anime shows and providers so we don't perform searches with a provider that doesn't support the type of show genre we are searching.
2014-05-27 03:09:57 -07:00
echel0n
7391a6e454 App name change in code. 2014-05-23 05:37:22 -07:00
echel0n
736cb7c738 Fixes issues with ABD shows searching for complete seasons and returning zero results. 2014-05-22 22:02:49 -07:00
echel0n
fa11b4ef3a Removed some sleep timers to improve overall speed of sickrage. 2014-05-20 11:01:51 -07:00
echel0n
c350c0fdf9 Fixed dupe issues with backlog searches.
Set cache to store results for 1 week and remove anything older then 1 week only if daily search has searched the results before hand.
2014-05-18 08:33:31 -07:00
echel0n
a3e06b965f Added in CPU Throttling option under general config -> advanced.
Fixed issues with passwords and unicode chars.
Fixed issue with usenet_retention variable not being saved or set.
2014-05-17 04:40:26 -07:00
echel0n
f9052ca2b2 Fixes to help with brining down the cpu usage of SickRage 2014-05-17 02:27:17 -07:00
echel0n
e96035ff4f Added per-provider options search mode and search fallback.
Search mode allows you to choose season packs only or episodes only which controls how backlogs search for data.

Search fallback if selected will attempt a 2nd search this time using the alternative search mode to see if it can get results if the previous attempt failed.

Provider settings are now read and stored dynamically instead of always having to retrieve them by there static names, this will allow for dropping in new provider modules easier to intergrate with existing options already present.

Fixed search code to not perform dupe searches by mistake.

Cache stores a list of shows we have rejected so that next time around we don't have to process them to know we don't like em.
2014-05-16 22:29:44 -07:00
echel0n
415e0df536 Fix for air-by-date and sports shows when searching for full seasons.
Fix for rssupdater errors
2014-05-14 01:01:36 -07:00