From b5d6c48a1680179e6d4c5711d97ccf36fc44266f Mon Sep 17 00:00:00 2001 From: stefanodvx <69367859+stefanodvx@users.noreply.github.com> Date: Wed, 23 Apr 2025 01:57:20 +0200 Subject: [PATCH 1/4] readme updated --- AUTHENTICATION.md | 9 +++++++++ README.md | 11 ++++------- 2 files changed, 13 insertions(+), 7 deletions(-) create mode 100644 AUTHENTICATION.md diff --git a/AUTHENTICATION.md b/AUTHENTICATION.md new file mode 100644 index 0000000..21323c7 --- /dev/null +++ b/AUTHENTICATION.md @@ -0,0 +1,9 @@ +# authentication +some extractors require authentication to access the content. simply export cookies from your browser in netscape format and place them in cookies folder (e.g. `cookies/reddit.txt`). you can easily export cookies using _Get cookies.txt LOCALLY_ extension for your browser ([chrome](https://chrome.google.com/webstore/detail/get-cookies-txt-locally/nhdogjmejiglipccpnnnanhbledajbpd) - [firefox](https://addons.mozilla.org/en-US/firefox/addon/get-cookies-txt-locally/)). + +extractors that **need** authentication: +- reddit +- twitter + +> [!CAUTION] +> using cookies _may_ be leading to account bans. we are not responsible for any bans or issues that may arise from using cookies. if you are using cookies, please make sure to use them responsibly and at your own risk. \ No newline at end of file diff --git a/README.md b/README.md index 9b37664..9807bd4 100644 --- a/README.md +++ b/README.md @@ -100,16 +100,13 @@ there are two types of proxying available: http and edge. **note:** by settings `NO_PROXY` environment variable, you can specify domains that should not be proxied. # authentication -some extractors require authentication to access the content. you can easily use cookies for that; simply export cookies from your browser in netscape format and place them in cookies folder (e.g. `cookies/reddit.txt`). you can easily export cookies using _Get cookies.txt LOCALLY_ extension for your browser. +some extractors require cookies to access the content. please refer to [this page](AUTHENTICATION.md) for more information on how to set up authentication for each extractor. # todo -- [ ] add more extractors -- [ ] switch to native libav - [ ] add tests -- [ ] improve error handling - [ ] add support for telegram webhooks -- [ ] switch to pgsql (?) -- [ ] better api (?) -- [ ] better docs with multiple readme +- [ ] switch to pgsql (maybe) +- [ ] better api +- [ ] better docs --- \ No newline at end of file From 3704e5cfc59f72582fb14a9d5f3a581e9f511e09 Mon Sep 17 00:00:00 2001 From: stefanodvx <69367859+stefanodvx@users.noreply.github.com> Date: Wed, 23 Apr 2025 01:59:39 +0200 Subject: [PATCH 2/4] Update AUTHENTICATION.md --- AUTHENTICATION.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/AUTHENTICATION.md b/AUTHENTICATION.md index 21323c7..640d483 100644 --- a/AUTHENTICATION.md +++ b/AUTHENTICATION.md @@ -1,5 +1,5 @@ # authentication -some extractors require authentication to access the content. simply export cookies from your browser in netscape format and place them in cookies folder (e.g. `cookies/reddit.txt`). you can easily export cookies using _Get cookies.txt LOCALLY_ extension for your browser ([chrome](https://chrome.google.com/webstore/detail/get-cookies-txt-locally/nhdogjmejiglipccpnnnanhbledajbpd) - [firefox](https://addons.mozilla.org/en-US/firefox/addon/get-cookies-txt-locally/)). +some extractors require cookies to access the content. you can export them from your browser in netscape format and place the file in cookies folder (e.g. `cookies/reddit.txt`). you can easily export cookies using _Get cookies.txt LOCALLY_ extension for your browser ([chrome](https://chrome.google.com/webstore/detail/get-cookies-txt-locally/nhdogjmejiglipccpnnnanhbledajbpd) - [firefox](https://addons.mozilla.org/en-US/firefox/addon/get-cookies-txt-locally/)). extractors that **need** authentication: - reddit From 244ebe1b1c8c687df2fb1d93a11c65c3a6f69545 Mon Sep 17 00:00:00 2001 From: stefanodvx <69367859+stefanodvx@users.noreply.github.com> Date: Wed, 23 Apr 2025 02:00:15 +0200 Subject: [PATCH 3/4] Update AUTHENTICATION.md --- AUTHENTICATION.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/AUTHENTICATION.md b/AUTHENTICATION.md index 640d483..d037669 100644 --- a/AUTHENTICATION.md +++ b/AUTHENTICATION.md @@ -1,5 +1,5 @@ # authentication -some extractors require cookies to access the content. you can export them from your browser in netscape format and place the file in cookies folder (e.g. `cookies/reddit.txt`). you can easily export cookies using _Get cookies.txt LOCALLY_ extension for your browser ([chrome](https://chrome.google.com/webstore/detail/get-cookies-txt-locally/nhdogjmejiglipccpnnnanhbledajbpd) - [firefox](https://addons.mozilla.org/en-US/firefox/addon/get-cookies-txt-locally/)). +some extractors require cookies to access the content. you can export them from your browser in netscape format and place the file in `cookies` folder (e.g. `cookies/reddit.txt`). you can easily export cookies using _Get cookies.txt LOCALLY_ extension for your browser ([chrome](https://chrome.google.com/webstore/detail/get-cookies-txt-locally/nhdogjmejiglipccpnnnanhbledajbpd) - [firefox](https://addons.mozilla.org/en-US/firefox/addon/get-cookies-txt-locally/)). extractors that **need** authentication: - reddit From 1b3c42680846f02a524e49baae42e1ab475f865e Mon Sep 17 00:00:00 2001 From: stefano <69367859+stefanodvx@users.noreply.github.com> Date: Wed, 23 Apr 2025 02:09:37 +0200 Subject: [PATCH 4/4] update readme --- README.md | 36 +++++++++++++++++------------------- 1 file changed, 17 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index 9807bd4..071fe94 100644 --- a/README.md +++ b/README.md @@ -18,16 +18,17 @@ this project was born after the discontinuation of a highly popular bot known as * [todo](#todo) # dependencies -- ffmpeg >= 7.x **(*)** -- libheif >= 1.19.7 -- pkg-config -- mysql or mariadb - -**note:** libav shared libraries must be installed on the system in order to build the bot. +* ffmpeg >= 7.x + * with shared libraries +* libheif >= 1.19.7 +* pkg-config +* sql database + * mysql or mariadb # installation ## build -_this method only works on linux and macos, if you want to build the bot on windows, check [docker installation](#docker-recommended) instead._ +> [!NOTE] +> there's no official support for windows yet. if you want to run the bot on it, please follow [docker installation](#docker-recommended) 1. clone the repository ```bash @@ -46,7 +47,6 @@ _this method only works on linux and macos, if you want to build the bot on wind ``` ## docker (recommended) - 1. build the image using the dockerfile: ```bash @@ -94,19 +94,17 @@ you can configure specific extractors options with `ext-cfg.yaml` file. document # proxying there are two types of proxying available: http and edge. -- **http proxy**: this is a standard http proxy that can be used to route requests through a proxy server. you can set the `HTTP_PROXY` and `HTTPS_PROXY` environment variables to use this feature. (SOCKS5 is supported too) -- **edge proxy**: this is a custom proxy that is used to route requests through a specific url. currenrly, you can only set this proxy with `ext-cfg.yaml` file. this is useful for routing requests through a specific server or service. however, this feature is not totally implemented yet. - -**note:** by settings `NO_PROXY` environment variable, you can specify domains that should not be proxied. +* **http proxy**: this is a standard http proxy that can be used to route requests through a proxy server. you can set the `HTTP_PROXY` and `HTTPS_PROXY` environment variables to use this feature. (SOCKS5 is supported too) +* **edge proxy**: this is a custom proxy that is used to route requests through a specific url. currenrly, you can only set this proxy with `ext-cfg.yaml` file. this is useful for routing requests through a specific server or service. however, this feature is not totally implemented yet. +> [!TIP] +> by settings `NO_PROXY` environment variable, you can specify domains that should not be proxied. # authentication some extractors require cookies to access the content. please refer to [this page](AUTHENTICATION.md) for more information on how to set up authentication for each extractor. # todo -- [ ] add tests -- [ ] add support for telegram webhooks -- [ ] switch to pgsql (maybe) -- [ ] better api -- [ ] better docs - ---- \ No newline at end of file +* [ ] add tests +* [ ] add support for telegram webhooks +* [ ] switch to pgsql (maybe) +* [ ] better api +* [ ] better docs