Login
Username:

Password:

Remember me



Lost Password?

Register now!

Sections

Who's Online
202 user(s) are online (75 user(s) are browsing Forums)

Members: 0
Guests: 202

more...

Support us!

Headlines

 
  Register To Post  

« 1 (2)
Anonymous
Re: Assigned project: Graphical Common Download Manager
@Hans

Thank you. Yes, I hadn't thought of that. It shoudl be possible to pass a list of headers and values including user agent string, cookies et al. So the browser, when it makes the call, passes over the URL AND the header information.

Thanks!

Go to top
Anonymous
Re: Assigned project: Graphical Common Download Manager
@xeron

Thanks Xeron, I don't mind at all!

Go to top
Re: Assigned project: Graphical Common Download Manager
Just can't stay away
Just can't stay away


See User information
@DaveP

Quote:
Its just about downloading files for browsers like OWB or NetSurf, not about keeping software up to date.
It can be a replacement for old download managers like HTTPResume, Charon, etc., but an external one can't be used by browsers like OWB.

Go to top
Anonymous
Re: Assigned project: Graphical Common Download Manager
@joerg

OWB can't send a request on a message port or call a library? Flip.

Anyhow, the source is completely open, so when its done, it can be integrated with whatever.

Go to top
Re: Assigned project: Graphical Common Download Manager
Amigans Defender
Amigans Defender


See User information
@DaveP

Quote:
Thank you. Yes, I hadn't thought of that. It shoudl be possible to pass a list of headers and values including user agent string, cookies et al. So the browser, when it makes the call, passes over the URL AND the header information.


I don't think I can do that with NetSurf. Maybe I might be able to reconstruct it. I'm not sure how much information is available at the point of starting the download.

Writing a download manager which can be slotted in to the browser's code might be more beneficial (and probably easier, given all the TCP/IP stuff is already done).

Go to top
Anonymous
Re: Assigned project: Graphical Common Download Manager
@Chris

Tell me what you need from me to achieve that, and I'll try to meet it.

Go to top
Re: Assigned project: Graphical Common Download Manager
Just popping in
Just popping in


See User information
@DaveP

Yes I would imagine it would be good to be able to have an api that could be slotted in to a browser (or for that matter any other application) so you could see all your downloads in a single location if you wanted (and obviously set download locations etc), but also provide a filtered list specific to the application.

As you say though it would need more details from the browser writers.

Mark

For custom designed applications, websites, ecommerce and secure hosting visit:

www.icestarmedia.com
Go to top
Re: Assigned project: Graphical Common Download Manager
Just can't stay away
Just can't stay away


See User information
@DaveP

Quote:
OWB can't send a request on a message port or call a library?
It would only work for downloading from sites like os4depot, aminet, etc., which don't have any restrictions, but on sites like rapidshare.com even passing the cookies and referrer to the download manager doesn't help since you can't download the same link twice, but that's what would happen with an external download manager (the browser starts the transfer, detects it's a download, aborts it's own transfer and passes the data to the download manager which starts the download again but only gets an error from the server instead of the file).

Quote:
Anyhow, the source is completely open, so when its done, it can be integrated with whatever.
Integrating a general download manager in OWB would probably be as much work as implementing an own one from scratch, and a lot of the download manager code, for example everything network related, would be useless.

Go to top
Re: Assigned project: Graphical Common Download Manager
Just can't stay away
Just can't stay away


See User information
@Hans

Quote:
Looking at the server logs of my website, OWB seems to use WGet for file downloads, which won't work with files that require authentication.
OWB doesn't use an external tool like wget for the downloads.

What's currently not supported in the AmigaOS 4.x port of OWB (except for version 3.8) is sending cookies set by JavaScript, and that's very likely the reason for your problems. For the next version I've removed the AmigaOS 4.x specific cookie changes, OWB's own cookie support finally seems to work with most sites. There may still be some problems, but I guess more will work that way than with OWB <= 3.15.

Go to top
Anonymous
Re: Assigned project: Graphical Common Download Manager
@joerg

Presumably you can pass a socket descriptor.

Go to top
Re: Assigned project: Graphical Common Download Manager
Amigans Defender
Amigans Defender


See User information
@DaveP

Easy place to start is to look at my existing code, you can do what you want within the framework.

http://source.netsurf-browser.org/tru ... ga/download.c?view=markup

Ignore anything below ami_free_download_list (the download list is a bit of a hack for ARexx, because if you send several download requests in quick sucession they tend to get dropped)

Basically it's one function to create a window, one which dumps the data to disk and one which cleans up. The event code is in a different file as my current window only has one button. Elsewhere in NetSurf I've put the event code for each window in the same file and it gets called from the main loop when the window type is determined (see amiga/gui.c and amiga/tree.c for example)

*shrug* I'm not sure if this answers the question?

Go to top
Re: Assigned project: Graphical Common Download Manager
Just can't stay away
Just can't stay away


See User information
@DaveP

Quote:
Presumably you can pass a socket descriptor.
No. I could pass a pointer to the OWB ResourceHandle object, which requires rebuilding the download manager for new OWB versions if something changes in it, or maybe a libcurl handle, but nothing more low level. In any case it couldn't be a general download manager any more but would be one which only works with OWB, and for something like that it wouldn't make any sense to create an external download manager since it would be much less work to implement it in OWB itself.

To pass a C library, or a bsdsocket.library, socket descriptor from OWB to a download manager someone would have to reimplement the OWB/WebKit network code using C library, or bsdsocket.library, sockets first. But without using a library like curl or soup that would be much more work than implementing a download manager in OWB as well ...

Go to top
Re: Assigned project: Graphical Common Download Manager
Home away from home
Home away from home


See User information
@joerg

Quote:

joerg wrote:
@Hans

Quote:
Looking at the server logs of my website, OWB seems to use WGet for file downloads, which won't work with files that require authentication.
OWB doesn't use an external tool like wget for the downloads.

What's currently not supported in the AmigaOS 4.x port of OWB (except for version 3.8) is sending cookies set by JavaScript, and that's very likely the reason for your problems. For the next version I've removed the AmigaOS 4.x specific cookie changes, OWB's own cookie support finally seems to work with most sites. There may still be some problems, but I guess more will work that way than with OWB <= 3.15.


Good to hear.

What's confusing me a little is that, in my logs I see the same IP address that is using OWB switching to WGet, and it doesn't look like someone is doing this manually (someone logs in first using OWB, and then I get the WGet download request). Ah, well; so long as it's fixed, it doesn't matter.

Hans

Join the Kea Campus - upgrade your skills; support my work; enjoy the Amiga corner.
https://keasigmadelta.com/ - see more of my work
Go to top
Re: Assigned project: Graphical Common Download Manager
Home away from home
Home away from home


See User information
@joerg

Quote:

joerg wrote:
@DaveP

Quote:
Presumably you can pass a socket descriptor.
No. I could pass a pointer to the OWB ResourceHandle object, which requires rebuilding the download manager for new OWB versions if something changes in it, or maybe a libcurl handle, but nothing more low level. In any case it couldn't be a general download manager any more but would be one which only works with OWB, and for something like that it wouldn't make any sense to create an external download manager since it would be much less work to implement it in OWB itself.

To pass a C library, or a bsdsocket.library, socket descriptor from OWB to a download manager someone would have to reimplement the OWB/WebKit network code using C library, or bsdsocket.library, sockets first. But without using a library like curl or soup that would be much more work than implementing a download manager in OWB as well ...


NPAPI provides a method of accessing streams. I don't know if it's flexible enough for this task, but, if it enables the download manager to resume a download, then it should be usable.

Alternatively, you, Chris and DaveP could decide on a common API that hides the actual socket implementation. A small set of functions should do. That way the download manager wouldn't care about the contents of socket pointers, etc. (call them socket handles in this case, because you don't know what they are).

Hans

Join the Kea Campus - upgrade your skills; support my work; enjoy the Amiga corner.
https://keasigmadelta.com/ - see more of my work
Go to top
Anonymous
Re: Assigned project: Graphical Common Download Manager
Chris

That is exactly what I need to see, thank you.

Joerg

I must admit even though in places I have used libcurl et al I have always much preferred doing my own HTTP implementations on top of sockets. I'll have a look at your source and see what is posible.

If a common module that /could/ be used could be developed (not asking either of you to pitch in, you've got your hands full I'm sure) that implemented both a download layer and a libcurl style replacement and provided a layer for other porting efforts/products as well as for script automation then I'll do it. Otherwise I'll just do it as an exercise in providing functionality to the AmigaOS desktop with a text entry field at the top (and an AREXX interface) that allows you to download a URL.

But I'll need to pick through what OWB is doing, and what Netsurf is doing to understand what I *cant* do.

Go to top
Anonymous
Re: Assigned project: Graphical Common Download Manager
I guess where I am now heading with this is that there are two, perhaps three modes of operation. A very basic illustration of what I mean:

1. A wrapper to various protocols. After connecting to the GDM you call:
DM_OpenURL( strptr url, <header list>, MODE_QUIET ) -> DMResponse *

struct {
int returncode;
<response header list> headers;
DM_Resource in;
DM_Resource out
} DMResponse;

Then once you have checked the return code, to read and write data you can do the following:

DM_WriteBytes( DMResponse * handle, char * buffer, int bytes ) -> int bytes written
DM_ReadBytes( DMResponse * handle, char * buffer, int bytes ) -> int bytes read

Followed by DM_Close( DMResponse * handle );

If MODE_QUIET is not set, it is visible in the download manager just as a tracking bar and a "stop" button, as the control is not really handled by the download manager.

2. The original suggestion which permits download automation, resume et al.

Basically they would use the same code underneath. But wrap the sockets layer.

Obviously I still need to look at the needs of either browser,..

Go to top
Anonymous
Re: Assigned project: Graphical Common Download Manager
Then there is the third mode of operation, which should possibly be handled internally. That is where various mime types are registered as being "download to download area" or if disposition is attachment.

Given the handler would read the headers before passing them back, the browser would essentially "let go" of that resource and allow the download manager to dump it to a file, or to an external media player.

So after you connect to the DM, in order to handle your I/O, you specify a list of mime types that YOU are prepared to handle. The rest relies on its prefs file.

Go to top
Anonymous
Re: Assigned project: Graphical Common Download Manager
Adding a fourth mode of operation, universal listener on sockets.

You ask it to listen, and pass all requests your way.
You get notified of requests, and data.
.

Go to top

  Register To Post
« 1 (2)

 




Currently Active Users Viewing This Thread: 1 ( 0 members and 1 Anonymous Users )




Powered by XOOPS 2.0 © 2001-2023 The XOOPS Project