Login
Username:

Password:

Remember me



Lost Password?

Register now!

Sections

Who's Online
24 user(s) are online (14 user(s) are browsing Forums)

Members: 3
Guests: 21

MigthyMax, walkero, walkerotest, more...

Support us!

Headlines

 
  Register To Post  

cURL - Downloading a site
Home away from home
Home away from home


See User information
Can anyone give any hints on how i could download a specific site with cURL?

Could it be a leech/DDoS protection?
I know it works with other sites, so maybe nothing can be done about it as it's site related?

The site is https://www.accuweather.com
cURL version is
Quote:
curl 7.79.1 (powerpc-unknown-amigaos) libcurl/7.79.1 OpenSSL/1.1.1l zlib/1.2.11 librtmp/2.3
Release-Date: 2021-09-22
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS HSTS HTTPS-proxy libz NTLM SSL TLS-SRP


cURL https://www.accuweather.com -o ram:test.html

is just sitting there counting seconds...

Thank you

Go to top
Re: cURL - Downloading a site
Just popping in
Just popping in


See User information
Hi,

Doesn't work with online tools also.
It works with the url : https://api.accuweather.com/ but I don't know if it's what you're looking for.

Regards,

Go to top
Re: cURL - Downloading a site
Home away from home
Home away from home


See User information
@Petrol

Bad Request - Invalid Header

HTTP Error 400. The request has an invalid header name.

edit:

Ah i see, that's the dev center for AccuWeather...nope, unfortunately not what i'm looking for


Edited by Raziel on 2022/8/25 17:32:29
Go to top
Re: cURL - Downloading a site
Just popping in
Just popping in


See User information
@Raziel

Users on the IBrowse mailing list have noted that cURL and Wget both use old versions of SSL that no longer work with many websites, so perhaps that's the problem. Maybe try the verbose (-v) option and see if that provides any clues.

Go to top
Re: cURL - Downloading a site
Home away from home
Home away from home


See User information
@msteed

Quote:

APPDIR:cURL -v https://accuweather.com -o test.html
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 2.16.104.230:443...
* Connected to accuweather.com (2.16.104.230) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
* CApath: /DEVS/AmiSSL/Certs
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [35 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [3935 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=US; ST=Pennsylvania; L=State College; O=Accuweather, Inc.; CN=*.accuweather.com
* start date: Aug 18 00:00:00 2022 GMT
* expire date: Sep 4 23:59:59 2023 GMT
* subjectAltName: host "accuweather.com" matched cert's "accuweather.com"
* issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS RSA SHA256 2020 CA1
* SSL certificate verify ok.
} [5 bytes data]
> GET / HTTP/1.1
> Host: accuweather.com
> User-Agent: curl/7.79.1
> Accept: */*
>
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* old SSL session ID is stale, removing


Does that tell anyone anything?

Go to top

  Register To Post

 




Currently Active Users Viewing This Thread: 1 ( 0 members and 1 Anonymous Users )




Powered by XOOPS 2.0 © 2001-2024 The XOOPS Project