Help with Cloudflare scraping
#1
Hello everyone,

First time poster, long time user. I've recently been getting my hands under the hood trying to fix some issues I've been having with the Catch-Up TV addon.
I've had some small success, but I've been banging my head on an issue for the past week, and I think it's time for me to seek some help.

So here's the issue :  Basically, some channels on the addon won't play back because the urlquick requests returns a 403 error. In reality, the addon is trying to request a regular website to parse the html content and locate the live and replay links, and those site are cloudflare protected.

So i've been trying to implement some of the cloudflare scraping solution in a way that would be fully Kodi integrated, without some external dependency outside the official kodi repo.

Using things like cfscrape seem to work in development, but fail once used in kodi, because of a JAVASCRIPT non enabled issue.

So here's my question :
Is there an already known way to parse cloudfrlare protected website and get their html from within Kodi ?
Or is there a way to make Kodi javacript enabled within its addons ?

Thank you for any help you might provide
Reply
#2
is it possible cloudflare is simply blocking the user-agent of Kodi? if so then you would just need to fake a different user agent

no, Kodi does not do anything even remotely related to Javascript despite many requests to modernize and add it
Reply
#3
I have been trying manipulating the request with every user-agent and other headers I could think of to get it to think I was a legitimate browser, but there's a js puzzle to solve, which is implemented in scripts like cfscrape ou cloudscrape, but they use js2py which I can't seem to import.

So that's where I'm currenlty stuck.

I don't know if it's possible to import a module in an addon.xml that isn't part of the official kodi repo (like a github link ?)
Reply
#4
(2024-03-05, 18:45)IceFreak Wrote: I don't know if it's possible to import a module in an addon.xml that isn't part of the official kodi repo (like a github link ?)

not from addon.xml but you can simply include it within your code structure, a lot of developers do

matthuisman includes js2py in his slyguy.dependencies addon which he uses for all his addons - https://github.com/matthuisman/slyguy.ad...es/modules
Reply
#5
Thank you for your help izprtxqkft, I used your advice and managed to make the cnews replay work on the catch-up tv addon Smile
Reply

Logout Mark Read Team Forum Stats Members Help
Help with Cloudflare scraping0