Commit graph

84 commits

Author SHA1 Message Date
stefanodvx
b5c5801ec2 fix: better slices allocation
pre-allocate slices' size when possible when extracting media
2025-04-28 15:57:30 +02:00
stefanodvx
0d986d4573 fix: resolve memory leaks in download utility
1. writing chunks directly to disk instead of buffering in memory
2. using fixed-size buffers (32KB) for all I/O operations
3. optimizing buffer allocation strategy in downloadInMemory
4. implementing proper file synchronization with mutex locks
5. calculating chunk boundaries on-the-fly instead of pre-allocating

the memory profiling showed excessive allocations in bytes.growSlice
which has been addressed by minimizing intermediate buffers and
eliminating unnecessary memory copies

these changes should fix the observed OOM issues when downloading
large files while maintaining the same functionality
2025-04-28 14:35:22 +02:00
stefanodvx
37c2fbf215 twitter: update endpoint
this endpoint is not protected at the moment, but would probably in a few days
2025-04-27 20:59:35 +02:00
stefanodvx
a959807524 twitter: temporary fix
we're aiming to automatically generate x-client-transaction-id header. due to a recent change on twitter backend, this header is now required. sadly, its not static, so would propably break again anytime soon
2025-04-26 19:50:30 +02:00
stefanodvx
1b7ed51060 threads: fixed captions 2025-04-24 12:29:42 +02:00
stefanodvx
effd834a47 threads: new extractor 2025-04-24 12:21:12 +02:00
stefanodvx
5336968e05 code cleanup 2025-04-24 12:20:35 +02:00
stefanodvx
917be1687f fixed typos 2025-04-24 00:34:58 +02:00
stefanodvx
491867d7e4 improved docs 2025-04-24 00:25:04 +02:00
stefanodvx
a24d9348bf instagram: fixes url pattern 2025-04-23 19:12:38 +02:00
stefanodvx
93e964a28b instagram: new extraction method
new extraction method first tries to fetch content directly from instagram graphql API, fallback to html embed page. in case every method fail, rely on 3rd party
2025-04-23 18:52:05 +02:00
stefano
1b3c426808
update readme 2025-04-23 02:09:37 +02:00
stefanodvx
244ebe1b1c Update AUTHENTICATION.md 2025-04-23 02:00:15 +02:00
stefanodvx
3704e5cfc5 Update AUTHENTICATION.md 2025-04-23 01:59:39 +02:00
stefanodvx
b5d6c48a16 readme updated 2025-04-23 01:57:20 +02:00
stefanodvx
c8d0666d1d fixes (desc)
Some checks failed
telegram message / notify (push) Has been cancelled
- m3u8 parser
- segments download
- video-audio merge
2025-04-23 01:44:25 +02:00
stefanodvx
330cc39583 trying to fix segmented download 2025-04-22 13:00:53 +02:00
HappyLoLTroll
1f618201f9
fix docker build (#7) 2025-04-22 12:14:39 +02:00
stefanodvx
4b1c801370 fixed typo 2025-04-22 12:12:42 +02:00
stefanodvx
c0c2e42436 added new instance 2025-04-22 12:11:20 +02:00
stefanodvx
34827fe852 reddit: prevent infinite loop on error 2025-04-22 12:06:13 +02:00
stefanodvx
7e69320d2c db: connection retry logic 2025-04-22 00:22:27 +02:00
stefanodvx
3e307658fa instagram: fixes share urls 2025-04-21 22:30:52 +02:00
stefanodvx
34219a848e instagram: fixes 2025-04-20 13:17:30 +02:00
stefanodvx
7dab9207b7 Update README.md 2025-04-20 13:10:17 +02:00
stefanodvx
e1f424b948 cleanup code 2025-04-20 13:03:53 +02:00
stefanodvx
12ad427baf fixes config 2025-04-20 12:54:20 +02:00
stefanodvx
6151a6c397 fix typo 2025-04-20 12:19:54 +02:00
stefanodvx
0a63df9ce6 set configuration for each extractor 2025-04-20 12:19:14 +02:00
stefanodvx
6baa965534 fix typos and remove unused headers 2025-04-20 01:26:50 +02:00
stefanodvx
0786312feb instagram: fix public instance with edge proxy 2025-04-20 01:07:02 +02:00
stefanodvx
6f5e792b01 edge proxy support + updated readme 2025-04-20 01:00:50 +02:00
stefanodvx
8d57b97b0f media: fixes default format filtering 2025-04-19 21:38:39 +02:00
stefanodvx
ebba6a8835 redgifs: new extractor 2025-04-19 12:52:07 +02:00
stefanodvx
f8b1cb6d48 fixes wrong url parse from message 2025-04-18 18:53:39 +02:00
stefanodvx
60365973d8 9gag: new extractor 2025-04-18 12:48:37 +02:00
stefanodvx
3b625c8b0a util: impersonate chrome TLS fingerprint 2025-04-18 12:46:26 +02:00
stefanodvx
ea1451cb97 libav: disable logging 2025-04-18 01:10:59 +02:00
stefanodvx
88e9bef29e remove unused packages 2025-04-18 00:26:51 +02:00
stefanodvx
52907ae952 Update Dockerfile 2025-04-17 16:48:26 +02:00
stefanodvx
55262d430d Update Dockerfile 2025-04-17 16:28:39 +02:00
stefanodvx
d92f02d38e implement libav in some methods 2025-04-17 16:21:59 +02:00
stefanodvx
9489689dbf Update notify.yml 2025-04-17 14:41:32 +02:00
stefanodvx
caaf055e58 Update notify.yml 2025-04-17 14:39:03 +02:00
stefanodvx
49cbd39a88 Update notify.yml 2025-04-17 14:36:40 +02:00
stefanodvx
92f87ea4bc send telegram message on push 2025-04-17 14:34:03 +02:00
dogghi no
e35bd2fe57
implement sonic for json + remove some TODOs from readme + fix go version requirement (#5) 2025-04-17 14:16:30 +02:00
dogghi no
4a15dd0761
replace mutex with atomic ops to reduce syscalls (#4) 2025-04-17 10:33:36 +02:00
stefanodvx
3848ff086b fixes #3 2025-04-17 10:19:01 +02:00
stefanodvx
7d19f8af71 misc: cache for cookies 2025-04-17 01:53:10 +02:00