Hi all! I’d like to introduce Snork, a tool I worked on alongside the DumbPad editor last week.
What is Snork?
Snork is a utility for monitoring and logging function calls to system libraries on AOS4. It’s designed for developers and advanced users who need to debug applications or analyze system behavior.
Years ago, on AmigaOS 3, I loved using Gerson Kurz’s Snoopy from 1994 (over 30 years ago!), and I appreciated its script-driven approach, which let me create specific scripts for each debugging task, combining functions from different libraries as needed, etc. However, the original Snoopy was written in 68k assembly, using SetFunction to patch jump tables, and isn’t compatible with AmigaOS 4 of course. So, I created Snork, which adopts a similar script-driven approach but is written in C, using IExec->SetMethod to patch modern library interfaces. It’s not a direct clone of course, but it borrows the idea of script-based control while being built from scratch for AmigaOS 4.
While AmigaOS 4 has tools like glSnoop by Capehill (focused on ogles2.library and warp3dnova.library) and Snoopy by Colin Wenzel (focused on dos.library with a GUI), these are limited to specific libraries. Snork, on the other hand, monitors function calls to system libraries like dos.library, intuition.library, exec.library, and others, with plans to support most system libraries in future versions. In its first version, Snork supports only dos.library and intuition.library and does not handle functions with variable arguments (e.g., tags or ...), support of which and additional libraries is planned for the next version.
Snork reads a configuration script called main.snork to determine which libraries and functions to monitor. Here’s how it operates:
Reads the Script: Parses main.snork to identify libraries (via base= lines) and functions to intercept (via watch= lines),along with logging templates. Patches Functions: Replaces function pointers in library interfaces with wrappers that log parameters before calling the original functions. Logs Parameters: Outputs logs to the serial port using IExec->DebugPrintF. You’ll need a terminal or a tool like Sashimi to view them. Handles Events: Runs in the background, responding to Ctrl+C to stop and clean up resources.
(click open in new tab for fullsize):
This screenshot shows a main.snork script example, the log output, demonstrating how it monitors function calls and part of Snork's guide.
Configuring Snork
Snork is controlled through the main.snork script, where you define libraries and functions to monitor. The scripts directory includes pre-written templates for all supported functions in dos.library and intuition.library. You can copy specific lines into main.snork and edit them as needed.
NOTE: Using all templates at once may slow things down due to heavy serial output, so select only what you need. For fun, you could include everything and watch the serial port get flooded with output!
The script uses two commands at moment:
1. base
Specifies a library and its alias.
Syntax: base=<alias>,<library_name>
alias mean short name (e.g., dos) and library_name mean full name (e.g., dos.library).
Example: base=dos,dos.library
2. watch
Defines a function to monitor and its logging template.
Syntax: watch=<base>,<function>,<template>
base mean library alias, function mean function name (e.g., Write) and template mean Parameter list, e.g., fh=%lp,buf=%lp,len=%ld.
After creating a main.snork script (or using the prebuilt default one for initial tests), run Snork in the shell. You’ll see output like this:
As long as you don’t press Ctrl+C, your serial port will show the debug output you specified.
The archive includes a guide with detailed instructions, so check it out for more information.
Feedback and Future Plans
This is the first beta, and there’s room for improvement. Please share your ideas, findings, and suggestions. What needs to change? What features should be added? Which libraries should come next? (I’m thinking about exec.library, utility.library, and graphics.library.)
Snork has been tested on real hardware (X5000, Pegasos2) and emulated QEMU/Pegasos2, but bugs are possible, of course. If you find issues or have ideas about , report it all plz.
Sounds like a cool tool (love the name!). I like that you can decide which parameters of each function to snoop on, rather than having to see them all, whether you're interested in them or not.
Quote:
Please share your ideas, findings, and suggestions. ... What features should be added?
It would be nice to be able to see the return value from the patched calls, though that will be a bit more complex as you need to report twice, once before the call and once after.
It would also be nice if it showed the name of the program making the call like OS4 Snoopy does, so you can see which calls are made by the program being debugged (and once you can do that, the option to snoop only on a specified program would help filter out clutter from other programs).
A good deal of caution should be employed when displaying a string parameter. Since one of the uses of the program is debugging, you have to consider that in some cases string pointers may not be valid (illegal values other than NULL or -1), or may not point at a real NUL-terminated string.
Somewhere down the road a standalone GUI tool -- something like MPlayer-GUI -- could be created to build the script files and send them to Snork for processing. But that can wait until all the basics are working.
Quote:
Which libraries should come next? (I'm thinking about exec.library, utility.library, and graphics.library.)
A good deal of caution should be employed when displaying a string parameter. Since one of the uses of the program is debugging, you have to consider that in some cases string pointers may not be valid (illegal values other than NULL or -1),
Using a IExec->TypeOfMem() check could partially help: If it returns 0 it's not a virtual address mapped by the kernel, and accessing it will very likely cause a DSI exception.
Quote:
or may not point at a real NUL-terminated string.
Checking that isn't that easy, but possible as well. Requires using such checks for each MMU page, if it's mapped or not. On most, or maybe all, OS4 systems the MMU page size is 4KB.
Just when you CTRL+C Snork just show a simple "Snork ended" on Shell/CLI (and maybe to serial outoput too).
Added, now it says start/end on both serial and console.
@msteed Quote:
It would also be nice if it showed the name of the program making the call like OS4 Snoopy does, so you can see which calls are made by the program being debugged
Done, now output looks like this (some simple trace of few dos functions):
IDOS->Printf("Usage: %s [--help] [--script=<filename>] [--taskinfo=<yes|no>] [--show=<process name>]\n", argv[0]);
IDOS->Printf(" --help Show this help message and exit\n");
IDOS->Printf(" --script=<filename> Script file to parse (must end with .snork, default: main.snork)\n");
IDOS->Printf(" --taskinfo=<yes|no> Show task/process info (default: yes)\n");
IDOS->Printf(" --show=<process name> Log only for specified process name (default: all processes)\n");
IDOS->Printf("Note: For process names with spaces, quote the entire argument, e.g., --show=\"ELF Collector\"\n");
So you can 1), enable/disable task names, 2) use any script you wish (default main.snork), 3) have output from any task you want.
For string handling at minimum will do as Joerg says with IExec->TypeOfMem() , better than nothing (for now).
ps. Btw, added also new keyword for template %t - so you can output tags instead of pointer, and then output will looks like this:
For that had to create a base of the tags for every library (as they can cross, in hope they will not).
And i also move all stuff to IPC now, i.e. in generic_patch just now send the message and call to original, all loging/parsing happens in main's handleinput.
For string handling at minimum will do as Joerg says with IExec->TypeOfMem() , better than nothing (for now).
For dealing with buggy pointers to something that's not a NUL-delimited string, I was thinking of something simple, like limiting the length that's printed to something reasonable, so you won't potentially get thousands of random characters until it happens to encounter a NUL.
In addition, you could filter the string to turn unprintable characters into periods (or some other character), so you won't possibly print an escape code or control character that interferes with the display of the snooped data. That would require making a copy of the string, since you don't want to alter the original.
Quote:
Btw, added also new keyword for template %t - so you can output tags instead of pointer,
Nice; I hadn't even thought of that one. Looking forward to the next update!
For dealing with buggy pointers to something that's not a NUL-delimited string, I was thinking of something simple, like limiting the length that's printed to something reasonable, so you won't potentially get thousands of random characters until it happens to encounter a NUL.
In addition, you could filter the string to turn unprintable characters into periods (or some other character), so you won't possibly print an escape code or control character that interferes with the display of the snooped data. That would require making a copy of the string, since you don't want to alter the original.
For now i limited string to 256 chars, + checking on null and 0xffffffff, + checked on format characters so to printf them as it, without taking as formatting symbols, cheking on fancy characters and replace them to (so printable ASCII character only) , etc. So before show the string i just do this:
Should be enough imho even without IExec->TypeOfMem() ?
I think the idea behind using TypeOfMem() is to catch cases where the string pointer points to something that is not a valid RAM address, where trying to read the characters of the string will probably cause a DSI.
Of course, if the string pointer is invalid the function you're calling will probably cause a DSI itself as soon as you call it. Still, there's some value to Snork not causing the DSI, even if there's going to be one an instant later anyway.
If TypeOfMem() returns zero you can print the string pointer as a hex value, which lets the user see what value was passed and perhaps provides a clue as to what went wrong (assuming the ensuing crash doesn't take down the system).
Of course, if the string pointer is invalid the function you're calling will probably cause a DSI itself as soon as you call it. Still, there's some value to Snork not causing the DSI, even if there's going to be one an instant later anyway.
Or (another suggestion) Snork doesn't pass the invalid pointer but passes on a valid string like "DEADSTRING\0". Than at least no DSI , still the string could mess up other stuff. But still some additional steps before a potential crash.
to add support for international chars, for example the default ISO 8859-15 ones. Unprintable control chars on AmigaOS, in any supported 8 bit charset, are only 0-31 and 127-159.
Of course, if the string pointer is invalid the function you're calling will probably cause a DSI itself as soon as you call it. Still, there's some value to Snork not causing the DSI, even if there's going to be one an instant later anyway.
Some functions may work with invalid string pointers, depending on other arguments the string pointer may not be used at all by the function itself and if it isn't accessed there wont be a DSI. But Snork would cause a DSI trying to print it.
to add support for international chars, for example the default ISO 8859-15 ones. Unprintable control chars on AmigaOS, in any supported 8 bit charset, are only 0-31 and 127-159.
Good catch , thanks !
Btw, what is the best way to follow if we need to parse data in a patch ? At fist, i just doing simple parsing inside of the patch, and then print it before calling original function. Then after , i add some simple IPC via PutMsg() (from patch) /GetMsg() (in main).
At first i tried to simple send what i have to the main, but find out that strings and other data simple died offten (because patch is done, memory changes, but i tried to handle it in the main which is point on whatever else). Because of that i had to copy in the patch before putmsg whole set : libname, functionname, doing findtaks(null), copy strings and tags, and then send this instead. For copy i use IExec->CopyMem/IUtility->Strlcpy (that for libname, funcname and strings), and for tags i simple do simple loop copy.
The question i had now , is it correct way of doing things. I mean, for speed and for being correct at all. As i doing dynamic copy for each patching function, i do not need locks and mutexes, but not sure shouldn't i somehow protect it or not ?
The question i had now , is it correct way of doing things. I mean, for speed and for being correct at all.
I've been thinking about this, and since no one else has jumped in, here are my thoughts.
Typically, your current approach -- doing as little as possible in the patch, and postponing the formatting and output to Snork itself -- is the recommended way to handle such things. But I wonder if in this case your original approach -- doing everything, including the DebugPrintF(), in the patch -- isn't a better idea.
One drawback to the current approach is that logging the function call is asynchronous to the function call itself- by the time Snork gets the message and outputs the log, the call has already occurred. So if the call crashes due to a bad argument, the crash occurs before the call can be logged (and if the crash takes down the system, the log will never be written).
A bigger problem comes up when you start logging exec.library calls. I presume you allocate memory in the patch to hold all the copies of arguments, and memory allocation can break a Forbid() that might be in effect when Exec is called. It looks like that can be prevented by using the AVT_Wait, FALSE tag with AllocVecTags().
An even bigger problem is that some exec.library calls may be made from interrupts, and memory allocation can't be done from within an interrupt. (I'd guess that TypeOfMem() isn't safe to use from an interrupt, either.)
Calling DebugPrintF() from within the patch eliminates both of those problems, while also making the logging synchronous to the function calls. And there's no need to copy anything except strings (where you're also filtering out unprintable characters), and those could be buffered on the stack (though you're using the caller's stack, so you have to be careful how much you use).
As far as I'm aware, there's no public information on the internal workings of DebugPrintF(). But given its intended purpose, I'd speculate that it depends as little as possible on the rest of the system (and presumably is okay to use from within an interrupt), and likely writes directly to the serial port (in SERIAL mode) with no buffering, and so doesn't return until the entire string has been sent.
If this is in fact the case it will be slow (especially when logging a tag list), as it takes 87 us for every character written at 115.2 kbps (it should be much faster if writing to the debug buffer or if using Sashimi). But the advantages of doing it this way may outweigh the reduced speed.
Typically, your current approach -- doing as little as possible in the patch, and postponing the formatting and output to Snork itself -- is the recommended way to handle such things. But I wonder if in this case your original approach -- doing everything, including the DebugPrintF(), in the patch -- isn't a better idea.
I switched to IPC only because I started losing strings (some had garbage when they shouldn't). Maybe it was a side effect of a bug or something else, but once I began copying strings and tags in the patch and sending them via messages, the bug disappeared. But of course, this means I now have more code to handle, like first copying, then parsing, and overall it’s starting to be a bit too much, but still probably safer...
Quote:
One drawback to the current approach is that logging the function call is asynchronous to the function call itself- by the time Snork gets the message and outputs the log, the call has already occurred. So if the call crashes due to a bad argument, the crash occurs before the call can be logged (and if the crash takes down the system, the log will never be written).
A bigger problem comes up when you start logging exec.library calls. I presume you allocate memory in the patch to hold all the copies of arguments, and memory allocation can break a Forbid() that might be in effect when Exec is called. It looks like that can be prevented by using the AVT_Wait, FALSE tag with AllocVecTags().
Yes, for copying strings and tags, I use AllocVecTags, and currently my GenericPatch function and SendIPCMessage function look like this:
// Set task name and address only if taskName is provided
if (taskName) {
IUtility->Strlcpy(ipcMsg->taskName, taskName, sizeof(ipcMsg->taskName));
ipcMsg->taskAddr = taskAddr;
} else {
ipcMsg->taskName[0] = '\0'; // Set to empty string (equivalent to NULL)
ipcMsg->taskAddr = 0;
}
// Copy string parameters
ipcMsg->stringParamCount = 0;
for (int i = 0; i < MAX_PARAMS && paramTypes[i] && ipcMsg->stringParamCount < MAX_STRING_PARAMS; i++) {
if (paramTypes[i] == 's') {
const char *str = (const char *)args[i];
if (str) {
IUtility->Strlcpy(ipcMsg->stringParams[ipcMsg->stringParamCount], str, MAX_STRING_LENGTH);
ipcMsg->args[i] = (uint32)ipcMsg->stringParams[ipcMsg->stringParamCount];
ipcMsg->stringParamCount++;
}
}
}
// Copy tag lists
ipcMsg->tagItemCount = 0;
ipcMsg->tagStringParamCount = 0;
for (int i = 0; i < MAX_PARAMS && paramTypes[i]; i++) {
if (paramTypes[i] == 't') {
struct TagItem *tags = (struct TagItem *)args[i];
if (tags) {
int tagCount = 0;
int moreCount = 0;
while (tags[tagCount].ti_Tag != TAG_END && tags[tagCount].ti_Tag != TAG_DONE &&
tagCount < MAX_TAG_ITEMS && ipcMsg->tagItemCount < MAX_TAG_ITEMS &&
moreCount < 10) {
if (tags[tagCount].ti_Tag == TAG_IGNORE) {
tagCount++;
continue;
}
if (tags[tagCount].ti_Tag == TAG_MORE) {
if (tags[tagCount].ti_Data && moreCount < 10) {
tags = (struct TagItem *)tags[tagCount].ti_Data;
tagCount = 0;
moreCount++;
continue;
}
break;
}
if (tags[tagCount].ti_Tag == TAG_SKIP) {
uint32 skipCount = tags[tagCount].ti_Data;
tagCount += skipCount + 1;
if (tagCount >= MAX_TAG_ITEMS) break;
continue;
}
// Check if the tag is a string type
const struct TagInfo *info = findTagInfoForTag(tags[tagCount].ti_Tag, libName);
if (info && info->Type == 's' && tags[tagCount].ti_Data &&
ipcMsg->tagStringParamCount < MAX_TAG_STRING_PARAMS) {
// Copy string to tagStringParams
IUtility->Strlcpy(ipcMsg->tagStringParams[ipcMsg->tagStringParamCount],
(const char *)tags[tagCount].ti_Data, MAX_STRING_LENGTH);
ipcMsg->tagItems[ipcMsg->tagItemCount].ti_Tag = tags[tagCount].ti_Tag;
ipcMsg->tagItems[ipcMsg->tagItemCount].ti_Data = (uint32)ipcMsg->tagStringParams[ipcMsg->tagStringParamCount];
ipcMsg->tagStringParamCount++;
} else {
// Copy tag as-is
ipcMsg->tagItems[ipcMsg->tagItemCount] = tags[tagCount];
}
ipcMsg->tagItemCount++;
tagCount++;
if (ipcMsg->tagItemCount >= MAX_TAG_ITEMS) break;
}
if (ipcMsg->tagItemCount > 0) {
if (ipcMsg->tagItemCount < MAX_TAG_ITEMS) {
ipcMsg->tagItems[ipcMsg->tagItemCount].ti_Tag = TAG_END;
ipcMsg->tagItemCount++;
}
ipcMsg->args[i] = (uint32)ipcMsg->tagItems;
}
}
}
}
And then, after I copy it all, I have to reparse and print it, so it's like double the work...
But for now I'm facing another problem: 64-bit arguments for some dos.library functions. The issue is:
Currently, all my patching code, wrappers for functions, etc., are using uint32/int32. That's everywhere, and all arguments are treated like this too (bytes, words, etc., are treated as longs, so int32, no problems there; strings and pointers fit into int32 as well). But then, dos.library has six functions that allow some arguments to be int64:
Now, if I patch those functions with my generic patch, which expects all arguments to be int32, I'll just get bugs because an int64 takes up two int32 slots, causing memory shifts, crashes, and other issues. I see that I need to handle int64 as two int32s in all cases, but that means changing GenericPatch, SendIPCMessage, and the logger to always skip two int32s when encountering an int64, and it’s starting to feel like a mess.
If I change everything in the code (wrappers, GenericPatch, etc.) to use int64, I’ll run into problems with int32 arguments for the same reasons.
So, for now, I’m thinking about how to solve this one...
@kas1e It's an absolute no-go to allocate memory in patched functions, no matter if you use C library (for example malloc()/free()) or exec library (for example AllocVecTags()/FreeVec)()) memory allocation and free functions: It can break the Forbid() and Disable() states, cause priority inversion, endless loops (if you patch the memory allocation/free functions as well) as well as other dead locks. You have to pre-allocate all memory required for the IPC before patching the functions, and only a single, pre-allocated memory buffer per patched function isn't enough either: The (patched) functions might be called by different tasks at the same time.
and only a single, pre-allocated memory buffer per patched function isn't enough either: The (patched) functions might be called by different tasks at the same time.
How can i know how much buffers i need then if there can be any amount of diffent tasks calling for example allocvec() (probably lots at the same time) ?
@msteed About 64bit issue: i simple create patch_exceptions.c in which patch by hands functions which not fits into generic_patch(). At moment there just 6 from dos.library using mixed 32/64bits args, intuitions ones as far as i see all fits into 32bit way.
...endless loops (if you patch the memory allocation/free functions as well)...
To prevent that you have to call the original version of any patched function that's called from within the patch. That would also apply to CopyMem(), PutMsg(), etc. And of course DebugPrintF(), if you call that from the patch.
I switched to IPC only because I started losing strings (some had garbage when they shouldn't).
I don't understand what was going wrong there. I can see strings disappearing after the patched call returns, since the calling program would be free to delete or modify them, and anything stack-based would of course go away. But it doesn't seem like that should happen if DebugPrintF() is called before the patch returns (assuming I'm correct that DebugPrintF() doesn't return until the entire string has been sent).
Quote:
About 64bit issue: i simple create patch_exceptions.c in which patch by hands functions which not fits into generic_patch().
Sounds like a plan. There are some functions that return 64-bit values too, if you get to the point of reporting on return codes.
@msteed Also another reason to switch to IPC: if i just call DebugPrintF from patching function, it sometime can overlap output. With IPC it wait for message, and then print it.
News feed
OS4 Feedback forum
OS4Depot Feedback forum
AmiCygnix forum
ABC shell forum
AmiKit forum
Cinnamon Writer forum
CodeBench forum
Digital Universe forum
Dopus 5 forum
E-UAE forum
Gnash forum
Ibrowse forum
JAmiga forum
Odyssey forum
OWB forum
Qt forum
SmartFileSystem forum
Timberwolf forum
TouchDevice forum
TuneNet forum
Unsatisfactory Software forum
Donate
