Commit graph

324 commits

Author SHA1 Message Date
adam
41931b33a9 Fixes Torrent Leech daily search URL
Fixes TV torrents daily search no results
2014-08-13 17:15:25 +08:00
adam
22f3a2e41b Consolidate more provider code
Fix tvtorrents issue

Remove old providers
2014-08-13 05:52:59 +08:00
echel0n
4d7d85e21b Merge pull request #704 from adam111316/hdbits_cache
Fix for HDbits tvcache issue
2014-08-10 21:37:02 -07:00
Adam
66f962f89b Fix for tpb ABD shows 2014-08-11 08:10:24 +08:00
Adam
e19f1ee53e Fix for HDbits tvcache issue 2014-08-11 07:59:56 +08:00
echel0n
318379522b Merge pull request #692 from Supremicus/dev
Provider image fixes.
2014-08-07 22:11:26 -07:00
echel0n
aa09e88d5f Merge pull request #689 from adam111316/tidy
Tidying provider code
2014-08-07 22:10:40 -07:00
Supremicus
b5c703c893 Provider image fixes.
newznab.py - changed to use provider image from set gui interface (for
gui skin support).
rsstorrent.py - as per above, directory was wrong all together.
2014-08-08 09:17:31 +10:00
Adam
6ad5ccf966 HDbits fix 2014-08-06 19:58:10 +08:00
Adam
4b3a680bb3 Tidy provider code 2014-08-06 01:38:16 +08:00
echel0n
1908c74e96 Merge pull request #679 from adam111316/nyaa_consolidation
Remove unnecessary code from nyaatorrents provider and PEP8
2014-08-03 00:14:11 -07:00
echel0n
76011d4653 Merge pull request #680 from adam111316/animezb_consolidation
Animezb tidy proper code and PEP8
2014-08-03 00:14:03 -07:00
echel0n
493f9af02e Merge pull request #681 from adam111316/fanzub_tidy
Tidy fanzub proper code and PEP8
2014-08-03 00:13:55 -07:00
Adam
74fa44d14e Tidy fanzub proper code and PEP8 2014-07-31 18:18:12 +08:00
Adam
f94b0085f0 Animezb tidy proper code and PEP8 2014-07-31 18:11:46 +08:00
Adam
4728e19edf Remove unnecessary code from nyaatorrents provider and PEP8 2014-07-31 17:45:51 +08:00
echel0n
dbc30c43cb Switched the following providers to use HTTPS connections instead of standard HTTP, more secure. 2014-07-30 22:02:45 -07:00
echel0n
9761c6c6a3 Fixed issues with torrent blackhole download issues. 2014-07-30 21:31:54 -07:00
echel0n
b8b5947ae6 Fix for trakt.tv issues when adding/removing/syncing shows.
Fix for indexer mapper to return a result no matter what.
2014-07-30 16:33:28 -07:00
echel0n
e6c8982d2b Testing fix for torrent download issues from ezrss 2014-07-28 21:42:53 -07:00
echel0n
19a89d453c We now check if a torrent url links to a valid file before adding as a verified result to get snatched, this helps prevent issues when attempting to add torrent to client later on to find the url returned nothing resulting in a error. 2014-07-28 12:19:41 -07:00
echel0n
60205979a8 Merge pull request #667 from adam111316/anime_propers_3
Anime proper support
2014-07-27 07:17:55 -07:00
echel0n
9493a8f7e5 Added back in missing code for finding propers, was removed by mistake.
Fixed issue with torrent clients not snatching due to improper handling of torrent hashes/files.
2014-07-27 07:04:37 -07:00
Adam
fd9be53fcb Anime propers 2014-07-27 21:04:17 +08:00
echel0n
b13e72e0a3 Moved code for cleaning up cache folder to seperate function located in helper.py.
Added call to cache folder cleanup during SickRage main init.

Changed a error message in rss feed class to a debug message for when url returns no data.

Moved indexer api cache files to be placed under cache folder subfolder indexers.

Moved rss feed cache files to be placed under cache folder subfolder rss.
2014-07-27 04:58:14 -07:00
echel0n
14c354b551 Added proxy support to Indexer APIs.
Provider getURL and downloadResult functions now removed and replaced with ones from helpers.py to help slim the code down plus allow more better control over request sessions.

Removed TVTumbler code.

Fixed HDBits provider.

Fixed config settings that were ment to be booleans but instead where set as str or int, should help resolve random html errors.

XEM Refresh check re-coded.

NameParser code for creating show object has been changed to only attempt at the very end once its found the bestMatch result, helps on resources and performance.
2014-07-27 03:59:21 -07:00
echel0n
0e5fbbc5c3 Fix for incorrect show snatches 2014-07-24 21:29:03 -07:00
echel0n
f47734446d Fixed further json issues with speedcd provider.
Fixed several providers and ssl cert verify issues.

Air-by-date shows now automatically get there respective season/episode numbers during release parsing.
2014-07-24 11:16:59 -07:00
echel0n
b16ff81478 Fixes for thread naming issues where provider names are getting stacked on top each other per loop iteration.
Fix for speedcd no json data issue.

Fix for DB upgrade syntax errors caused by previous commit.
2014-07-24 09:12:29 -07:00
echel0n
161226180f Cleaned up NyaaTorrents search code and fixed unicode issues. 2014-07-24 01:33:38 -07:00
echel0n
a697805923 Fixed NameParser to properly parse anime episodes that use normal season and episode naming conventions.
NameParser now properly creates absolute numbers from season/episode info or season/episode numbers from absolute info for anime shows. Removed scene convert routine and migrated the code into the parsing routine with a boolean to scene convert if needed.

Fixed NyaaTorrents issue with dict object errors.
2014-07-24 01:27:35 -07:00
echel0n
de5db9be64 Added ability to set a default indexer for trakt notifier used for adding shows from watch list so that SickRage knows what indexer to set the show as.
Indexer mapping now uses indexer api calls to gather its information and then stores it to a new table called indexer_mapping for instant lookups later on.

Fixed trakt related issues for adding new shows and syncing.

Centered items at bottom of pages to just look a little nicer and fit properly.
2014-07-23 22:07:01 -07:00
echel0n
3a2b67330c During backlog/manual/failed searches we now cache disgarded/ignored results pre-parsed for usage later on incase we end up setting a episode to wanted that matches said results, allows for maximum performance and helps limit the waste of resources used. 2014-07-21 23:00:58 -07:00
echel0n
a317ff61c2 Created a context manager wrapper for BeautifulSoup4 so that we can cleanup/clear tags/context on exit via WITH statements.
Fixed issues with torrent providers returning no results.
2014-07-21 21:26:58 -07:00
echel0n
899d03b5fe Fixed issue with appending UNNOWN qualities to provider results after being sorted. 2014-07-21 18:48:16 -07:00
echel0n
bf41ba59be Fixed sorting provider results by quality code, was causing a error. 2014-07-21 17:55:45 -07:00
echel0n
403c267953 Further improved memory handling of bs4 for torrent providers. 2014-07-21 16:01:46 -07:00
echel0n
4a29476415 Added code to perform disposal of bs4 class when finished as well as perform garbage collection afterwards, this fixed a memory leak issue SR was experiancing during backlog/manual/failed searches. 2014-07-21 15:16:04 -07:00
echel0n
f340f9b073 Fix for sports shows. 2014-07-21 08:49:20 -07:00
echel0n
6f817c0c5c Quality sorting fixed for provider results.
Reverted newznab search strings being confined to just season till we can find a better way of correcting the issue with Usenet Crawler.
2014-07-21 08:38:56 -07:00
echel0n
39054d7810 Fix for validating episode data during provider result gathering. 2014-07-21 07:21:03 -07:00
echel0n
7b54611361 Moved code for marking failed download successful out of loop.
Newznab providers now create season search strings for all searches as single ep search strings combined with using rage id numbers seems broken currently.
2014-07-21 06:44:01 -07:00
echel0n
5ac99b8c5c Fix for extensions being stripped off by mistake made when adding in -RP fix from few commits ago. 2014-07-21 06:29:07 -07:00
echel0n
57cf86279b Places Quality UNKNOWN items from provider results at end of itemsList leaving order of sorted items highest to lowest. 2014-07-20 23:59:57 -07:00
echel0n
d6442e0685 Provider results get sorted by quality before filtering occures. 2014-07-20 23:43:52 -07:00
echel0n
af0ccd65cf Fixes for a few provider issues plus passing of search_mode variable. 2014-07-20 22:47:13 -07:00
echel0n
ff5107cfe2 Provider searches for backlog, manual, and failed have been re-worked to not hammer the providers so much plus perform alot faster. 2014-07-20 22:07:53 -07:00
marseneault
662353e936 Add support for Bitsoup 2014-07-20 12:12:22 +08:00
Justin Nielsen
ad17b81560 Added TVTorrents.ro Provider
Missed some files

Removed seeders==0 as per request
2014-07-20 11:09:26 +08:00
echel0n
d02c0bd6eb Fixed issues with editing/saving custom scene exceptions.
Fixed charmap issues for anime show names.

Fixed issues with display show page and epCat key errors.

Fixed duplicate log messages for clearing provider caches.

Fixed issues with email notifier ep names not properly being encoded to UTF-8.

TVDB<->TVRAGE Indexer ID mapping is now performed on demand to be used when needed such as newznab providers can be searched with tvrage_id's and some will return tvrage_id's that later can be used to create show objects from for faster and more accurate name parsing, mapping is done via Trakt API calls.

Added stop event signals to schedualed tasks, SR now waits indefinate till task has been fully stopped before completing a restart or shutdown event.

NameParserCache is now persistent and stores 200 parsed results at any given time for quicker lookups and better performance, this helps maintain results between updates or shutdown/startup events.

Black and White lists for anime now only get used for anime shows as intended, performance gain for non-anime shows that dont need to load these lists.

Internal name cache now builds it self on demand when needed per show request plus checks if show is already in cache and if true exits routine to save time.

Schedualer and QueueItems classes are now a sub-class of threading.Thread and a stop threading event signal has been added to each.

If I forgot to list something it doesn't mean its not fixed so please test and report back if anything is wrong or has been corrected by this new release.
2014-07-14 19:00:53 -07:00
echel0n
09f53d3537 Filtering of torrents with 0 seeders is no longer forced, filtering now is done strictly by min seed and min leech user settins which can be set from search provider config settings. 2014-07-12 16:16:58 -07:00
echel0n
21cc92664f Fix for KickAss Torrents Invalid URL error. 2014-07-11 21:12:25 -07:00
echel0n
b7ab95996c Merge pull request #638 from adam111316/add_animezb_provider
Add animezb provider
2014-07-11 19:40:51 -07:00
echel0n
2d87de0fd8 Fix for search provider config page errors. 2014-07-11 14:57:49 -07:00
adam
923a58b2ab Add animezb provider 2014-07-12 01:21:56 +08:00
echel0n
aa74dfb68d Merge pull request #637 from marseneault/torrentbytes
Add TorrentBytes provider
2014-07-11 10:18:08 -07:00
echel0n
0e3495d30f Fix for show parsing errors to be displayed via debug logs now instead as a warning. 2014-07-11 00:59:05 -07:00
root
6304c5ed17 Add TorrentBytes provider 2014-07-10 10:15:44 -04:00
echel0n
e891e9d637 Fixed nextaired not found issues and fixed kat provider issues. 2014-07-09 09:01:12 -07:00
echel0n
f925488373 KickAss torrents now uses multiple urls to try from incase one goes down. 2014-07-09 01:26:45 -07:00
echel0n
15c7332277 Merge pull request #623 from nielsenj/customrssratio
Adds custom RSS provider ratio setting.
2014-07-08 09:53:07 -07:00
echel0n
7a95b0bb14 Added new InvalidShowException to NameParser, Removed isGoodResult search check and now attached parsed result show objects and compare them against the show object we use to search with instead. 2014-07-06 06:11:04 -07:00
echel0n
a3a3777d33 Fixed blackAndWhitelist for anime release groups. 2014-07-06 04:12:16 -07:00
echel0n
56e2c28bad SickRage now uses a internal name cache comprised of (showlist, scene exceptions, scene names) used to search and create show objects from which is needed for parsing search results.
Scene exceptions now uses a internal cache for scene exceptions and scene season exceptions, helps reduce overhead to DB and performs faster lookups when making scene exception requests.
2014-07-03 10:30:15 -07:00
echel0n
f0146f728e Lowered CPU usage even further, re-wrote daemonizer code for startups, removed misc unrequired functions from providers. 2014-07-03 07:07:07 -07:00
echel0n
36e12a5d4d Fix for hdtorrents uid issues. 2014-07-02 13:39:27 -07:00
echel0n
ddd9376faf Fix for torrentday uid issues. 2014-07-02 13:38:05 -07:00
echel0n
d6431570f1 TPB season pack searches fixed 2014-07-02 13:06:29 -07:00
echel0n
12ac388dc2 Fixed startup/restart/shutdown issues on Windows, Linux, FreeBSD platforms tested.
Fixed for updating issues.

Fixed high cpu and memory usage.
2014-07-02 11:51:14 -07:00
nielsenj
6d8e79c78a Adds custom RSS provider ratio setting.
Adds ratio setting to the custom RSS provider.
2014-06-30 22:48:49 -07:00
echel0n
a1cef0b958 Added in context to aid in garbage cleanup to help reduce memory usage. 2014-06-30 10:48:18 -07:00
echel0n
75f69396d4 Reverted some changes. 2014-06-30 08:57:32 -07:00
echel0n
5e507b3849 More memory reductions and cleanups 2014-06-30 04:44:36 -07:00
echel0n
0c57676aed Updated tornado source code.
Created custom class for rss feed parser.
2014-06-29 03:05:33 -07:00
echel0n
e0eea79ec5 Fix for ABD and Sports shows issue with toordinal function. 2014-06-25 21:39:34 -07:00
echel0n
61e1e5d2c2 Fix for __exit__ DB errors 2014-06-21 18:35:57 -07:00
echel0n
7e0bb651b2 Database now closes connection after each query and reconnects if closed to ensure no more connection and locked database problems.
Fixed more internal 500 errors.
2014-06-21 15:46:59 -07:00
echel0n
7ce409e90d Merge pull request #612 from JackDandy/port_fix_BTN_duplicate_search_params
Port:Fix BTN adding same search parameters multiple times.
2014-06-19 09:47:45 -07:00
echel0n
1fa8686422 Merge pull request #610 from JackDandy/port_fix_omgwtfnzbs_findPropers
Port:Fix omgwtfnzbs findPropers.
2014-06-19 09:46:58 -07:00
JackDandy
ff524fc7e4 Port:Fix omgwtfnzbs findPropers. 2014-06-19 16:37:29 +01:00
JackDandy
42f018bf06 Port:Fix BTN adding same search parameters multiple times. 2014-06-19 16:36:13 +01:00
fittysix
c5045d0f1e Add moderator to confirmed torrent posters
I think we can trust these guys :)
2014-06-18 07:04:16 -06:00
echel0n
98ff924b1c Fix for nullhandler issues for py2.6 users,
Fix for rsstorrents.
2014-06-17 08:46:07 -07:00
echel0n
d73cc1cbbd Replaced cherrypy with tornado which helped resolve our memory leak issue. 2014-06-14 20:56:15 -07:00
Adam
5fd0a2a1f3 Fix for Newznab providers when searching for a show without a tvrage id we use the name of the show. If the show name is more than one word we are using a _ as our delimiter which works fine unless the NN provider uses sphinx.. as the underscore then is used as part of the show name.. thus we always get back no results. We should just use . as it works for both sphinx and non sphinx servers. Tested this out on nzbs.org / nzb.su (sphinx) / sbi (non sphinx). 2014-06-12 11:19:27 +08:00
Nils
166dffeb5a Merge pull request #582 from bikemike222/add_cookies_rss
Added cookie support to custom torrent provider
2014-06-09 00:20:37 +02:00
echel0n
a1da7df12e Fixes for DB issues 2014-06-07 15:35:16 -07:00
echel0n
d00d55fdfc Modified DB code to close its connection if right after its finished with it, helps performance-wise and should resolve locked db issues as well. 2014-06-07 14:32:38 -07:00
echel0n
24dfbc3f15 Quality is now set during parsing of results.
Fixed more anime regex bugs.
Fixed Indexer API issues for anime shows and unicode problems.
2014-06-07 04:06:21 -07:00
echel0n
05dcdd72d7 Fix for air-by-date/sports/anime provider searches 2014-06-06 16:55:14 -07:00
echel0n
cab05efa64 Added anime search support and removed extra pipe symbole from search string. 2014-06-06 15:27:20 -07:00
echel0n
14201c71f3 Fixes issues with scene numbering being set to 0x0 after snatch is performed.
Fixes issues with KAT Provider sphinx error on searches.
2014-06-06 15:16:15 -07:00
echel0n
4df31bccee PublicHD removed till further notice. 2014-06-06 13:48:25 -07:00
Nils Vogels
0e989fe90f Limiting search to English-translated only (for now) 2014-06-02 00:05:07 +02:00
echel0n
d7396896b5 Added in regex matching with fuzzy matching.
Shows now display Indexer absolute numbering.
Improved speed of parsing search results.
Fixed episode naming issues.
2014-05-31 03:35:11 -07:00
echel0n
997896a540 Fix for transactionals and None Types in sql lists 2014-05-30 04:42:31 -07:00
echel0n
f01c5852d4 Improved find propers code.
Backlog and manual searches now check cache first for search results.
2014-05-30 00:51:24 -07:00
echel0n
e009641804 SickRage now stores the XEM scene data to the main DB under tv_episodes for faster lookups and better linkages with epiosde objects, insures we have scene data on hand when we want it!
Adding functions to lookup scene season and ep info via scene absolute numbers.

We now try and create a show object from parsed release info before we consider it a valid parsed result to insure we even have the show in our show list, also used to validate anime releases.

Misc bug fixes here and there.
2014-05-28 14:16:03 -07:00
echel0n
34da3f53d8 Added Black and White list for release groups for anime shows.
Fixed anime show list splitter.

Added check for anime shows and providers so we don't perform searches with a provider that doesn't support the type of show genre we are searching.
2014-05-27 03:09:57 -07:00
echel0n
f7b11e1e98 Fixed issues with post-processing.
Fixed issue with priority post-processing, bug would allow 2nd attempt to go through even if priority settings were not selected.

Fixed issues with cache retrieval and storing of provider results, properly stored now with season and ep info converted to indexer numbering.

Fixed a bug that was caching unconverted scene numbered files for post-processing causing us to skip converting them when needed.
2014-05-26 03:42:34 -07:00