@kas1e
Quote:
Sorry, you completely misunderstood and mixed up what I wrote
I said that I artificially modified ogles2 for testing to cancel out every draw call with more than 20 triangles.
I did so to see if there is a performance difference. If ogles2 is the bottleneck in this scenery here then there should be a significant difference, because the draw-commands are where most of the time-consuming stuff inside ogles2 happens.
Since this is not the case, gl4es is very very likely the major bottleneck.
Where this bottleneck is, no idea. But I also told you that the batching is fine now, at least in terms of number of triangles sent.

Quote:
I know, that's the very nature of things. Glad I didn't ask such a question...

@utri007
Quote:
Most were made by BSzili, some were made by me, some we made hand in hand (I fixed some stuff in R200, he added the coresponding features to MiniGL).
2.21 is really a fat update and it is key to exploit some of the R100/R200 improvements made back then (e.g. texture compression, stencil effects).
The one most interesting to Raziel was some rather experimental support for glDrawPixels(GL_DEPTH_COMPONENT), which makes some ScummVM games working well.
BSZili committed that glDrawPixels(GL_DEPTH_COMPONENT) thing in March 2016 (I had no write access back then). It's just that we forgot to increase version number / update the readme back then.

Quote:
No.

@samo79
@broadblues
While 2.22 is a real micro fix, it's not sooo unimportant as it might look, because as being mentioned the bug probably caused side-effects in the mipmapping code (I didn't check it out in depths, I just saw that it got called from there).

@kas1e
back on topic
Great, some very nice boost with gl4es! And best of all: it was achieved without touching ogles2 or Nova, you just needed to feed it correctly and get rid of some slow code in gl4es
And from what you told the Pandora comparison was incorrect: it was even slower (2 fps) than the Amiga version if using the same code-path.

From my quick tests it looks as if the one eating most frame-time here is still gl4es though.
E.g. if I actually cancel out every draw call with more than 20 triangles now (so that just enough gets rendered to see the fps counter and being able to get through the menus etc.) then performance doesn't change for me significantly at all. But the thing is that practically all potentially time-consuming stuff inside ogles2 is only triggered if such a draw-call is actually executed.
I don't know what the concrete issues in gl4es are. More such malloc/free stuff could cause it. Or very cache-unfriendly behaviour. Both are things which can easily have much less impact on other systems.

I don't say that there isn't any optimization potential inside ogles2, but the by far major time-eater seems to be gl4es at the moment, so before I concentrate on squeezing out some more performance out of ogles2 (which is kind of very hard as long as the client is the major bottleneck) this should be accelerated.

That messed up display when using Q3's ext-drawing looks to me like corupt memory somewhere.
I will probably take a deep look at ogles2 today, maybe it's not "corupt" memory but rather "misinterpreted" memory and the internal client-RAM-emulation VBOs are copied together falsely. That's rather unlikely but not too unplausible from the symptoms. Or maybe the indices-VBO is falsely setup (considering that there was a recent change in that area in ogles2, maybe sth. got messed up there).

What speaks against all this is the fact that gl4es uses ogles2's glDrawElements all the time (and glDrawArrays uses pretty identical code too), e. g. when it translates glBegin / glEnd, but well, there's more to glDrawElements than the pure function call. It's mostly about pointer setup / data conversion and such. And this may look totally different in one render-path compared to the other. And there may well be an until now undetected issue inside ogles2.

I'll get back when I ruled in / out this.
Cheers,
Daniel

@kas1e
So apples and pears have been compared? Anyway, we are back to where we left some posts ago where I told you:
rest, give it a break
Wait until Hans finished his work on Nova, then wait until I checked and eventually finished fixing ogles2 if there's an issue despite Nova supporting mixed types of shader variables then.
Until then: let's put this here to sleep (at least I will, I won't get back here until then).

Cheers,
Daniel

@kas1e
your post and my edit overlapped, please see addon above for even more clarity

Yes, it may well be that W3DSI contains extra batching-logic. Or MiniGL does, dunno. That would explain why it isn't hurt by that. And that's why I say: put that logic into the MiniGL equivalent, which is gl4es.

@kas1e

Quote:
I say that making hundreds or thousands of glDraw-calls with 10 triangles per frame each is a niche task for ogles2/Nova.

Quote:
Absolutely true.
But that means something else than throwing hundreds or thousands of such micro-draw-calls per frame at the ogles2 / Nova and expect it to deliver fast results.

Quote:
I told you exactly what I mean above:
"So the obvious solution is: extend gl4es instead to collect the data of such small draw calls and then issue a bigger one"

This is the solution to your problem.

Quote:
I already told you my guess on that already too:
"Apparently you're lucky and other ogles2 implementations on other hardware isn't hurt by that so much."

Quote:
Too bad then, because, as being said above:
in gl4es "it's also most likely easier to do and other systems also benefit from it" (even if it already runs "fast enough" on others).
ogles2/Nova can draw scenes with hundreds of thousands of triangles per frame - if you feed it correctly. Right now you don't feed it correctly, simple as that.
And the way you incorrectly feed it right now is so uncommon to ogles2 applications that I don't see the need / value / work-value-balance to optimize ogles2 also for this (ab)use-case. It is sth. the caller should take care of. The caller is gl4es.

Btw.: I don't know Q3 sources but somehow I cannot imagine that the game itself doesn't produce larger batches of triangles.

Quote:
Actually it's not strange at all and consistent to your very first tests.
Essentially what Q3/gl4es is doing here is the same as if you would put a big for-loop around the begin-end block of your rotating cube example, instead of putting that loop *inside* the begin-end-block. So simply spoken, instead of sending 1x 1000 cubes you are sending 1000x 1 cube (actually Q3/gl4es is even worse, a cube has at least 12 triangles ). This is what I call a niche use / abuse of ogles2. Why should something like that produce better performance than the original 1 cube per draw-call per frame?
No, there's nothing strange at all. The only strange thing is that Q3/gl4es issues such draw calls in the first place.

@kas1e

Quote:
Why that? Apparently I wasn't clear enough

Right now Q3/gl4es draws a scene in a way that's no good with ogles2/Nova. The latter like rather big amounts of triangles. That's what they are designed for. And this is how you get good performance from it. (Not just) people like Entwickler-X demonstrate how it's done.

Making hundreds or thousands of draw-calls with less than 10 triangles each is missing the topic of those libs. And it was never a good idea. Apparently you're lucky and other ogles2 implementations on other hardware isn't hurt by that so much. And apparently you're lucky that MiniGL/Warp3D(SI) is of some help in the background.

The thing is, like said before, that this type of inefficient drawing is not what 99% of ogles2 programs do. That's why I'm absolutely not convinced that it makes sense to optimize ogles2 also for this niche task. IMHO something like that has to be implemented in the next higher level (where it's also most likely easier to do and where other systems also benefit from it). The next higher level in case of the constellation here would be MiniGL or its equivalent.

So the obvious solution is:
extend gl4es instead to collect the data of such small draw calls and then issue a bigger one.

So, it's only a "sad final" if you either don't want to do that or if you cannot convince me that I'm wrong and that I should extend ogles2 by such an optimization.

@kas1e

Quote:
Drawing single triangles pre glDraw-call? Certainly not. And if a game does so, bad luck, at least with ogles2

Quote:I already told you my guess: probably W3DSI does some batching internally. Or MiniGL itself, dunno.

But I know one thing for sure:
ogles2 is not optimized / designed for thousands of single-triangles-from-client-mem-draw-calls per frame, which is why we got that low performance here.
ogles2 is well optimized for what people usually do with it, while considering the way Nova likes it:
mostly use VBOs and if using client-mem-arrays at all, then those are usually not just a handful of triangles.
It is not optimized for what Q3+gl4es deliver right now and I probably won't optimize it for that kind of stuff.

Quote:Don't know.

Quote:It will, as soon as you start to feed it with something else than single triangles

@kas1e
Drawing a scene like that is the ultimate most inefficient way to do things and one of the big "donts" in terms of GL.

I suppose W3DSI does extra batching internally. I however am not certain yet if I shall add sth. like that to ogles2.lib as it's such an insane API abuse.

If MiniGL does it, then maybe it's sth. to add to gl4es, since it's the MiniGL equivalent here, right?

@kas1e
The reason for Q3 being so slow is that the game does practically zero batching.
ogles2 is flooded by glDraw-calls of practically always less or equal to 10 triangles.
If I artifically limit ogles2 to ignore any draw-calls with more than 10 triangles, then everything looks like before

@kas1e
Rest asured (and I mean rest ): Hans knows very well how important that is for gl4es.
The point is simply:
you should not modifiy gl4es. Just throw test-progs at us one by one, we'll fix the issues step by step as they appear.
The constant "monster workarounds" in gl4es are of exactly zero benefit, things won't get done faster by that and it won't reveal issues that won't get revealed anyway.
It's just a waste of time of you and the gl4es-guy and it may actually complicate things as it also may reveal "pseudo-issues" that in fact turn out to be only side-effects of already known issues.

@kas1e
Quote:
Which is something we knew before that already.
Really, why do you guys burn your time for a "monster workaround" which will be reverted anyway for sth. which we already found out and Hans already said he's taking care of?

@kas1e
Quote:Finished just in time. Half an hour later my FTP etc. would have gone to bed

Quote:Yes, adding such an option is no problem and doesn't hurt.

@Hans
Quote:Yes, I saw this in the logs too and already added that missing extension string.

Quote:ogles2 contains logic to only flush if necessary, because yes: I also found too many flushs to be the main performance killer back then.
But never say never : we'll see when I found the time to check it out.

@kas1e
Quote:The number of draw commands being issued is not directly coupled to any flushs (e.g. my boing-ball test prog issues 1024 glDraw-calls and performs well even if not using VBOs).

Quote:Yes, but they also use VBOs a lot, so I'm actually wondering why those would cause any problems.
On the other hand they use hand-crafted shaders that are designed with current Nova limitations in mind and for optimal performance, so maybe it's also just something at that area.

Quote:Does it? Since when?

Quote: Of course it does *not*. Don't you think I would have told you? MGLReloaded is currently good enough for about 80% of the demos that ship with MiniGL, that's it.

Quote:I actually find this observation here rather interesting. Sounds as if the shaders in use / number of textures being used has a very strong impact on performance here; actually one I wouldn't expect in that strength.

But well, as being said: we'll know more / for sure what's happening when I checked it out!
This may take some time though.

@broadblues
Yes, ogles2.library also has different ways how to specify the render target. Besides supplying a window-handle like in this case here you can also supply your own Bitmaps or tell the lib to manage a buffered screen for your.
Since recently you can also switch between all those methods on the fly without losing your context.

@kas1e
Quote:
Put more emphasize on the good news The rendering output looks almost perfect.

@Capehill
Quote:
Yes, that was my first thought too.

@kas1e / all
However, even without VBOs it shouldn't be that slow, at least not at the ogles2 / Nova level.
Each of the three involved libs (gle4es, ogles2, Nova) can be the cause for this, of course. It can probably also be some kind of shader problem, you cannot tell without some real measuring and lib-logging.

Therefore: please pack all your stuff (ready to run please) into a nice zip and put it on my FTP (don't try to mail me such a fat pile), so that I can profile and rule out / rule in / locate problems in ogles2 and shaders at least.

While I was typing this your GL_EXT_compiled_vertex_array report appeared.
Please put that build in the zip too.
VBOs and such are actually well tested in ogles2, so I think it's unlikely that there's an issue there, but we'll see and better make sure

Please don't expect results too quickly, I also have other things to do

@kas1e
Quote:
Note: that's no bug-fix, but a feature request for Nova!

Strictly spoken it's a specs violation in ogles2.library, because the GLES2 specs defined that behaviour, not the Nova specs!
Considering that Nova currently strictly enforces shader attributes variables to be of the very same type as the attributes sent by the client-app, it would actually be ogles2.lib's job to adjust such data before sending it to Nova.
However, the potential performance impact would be high if doing that in ogles2.lib. I originally thought about making it optional in ogles2, so that coders who write their shaders with strict type matching wouldn't get "punished".
But luckily Hans told me that it's practically for free inside Nova, that's why he was kind enough to take this on his shoulders.

Quote:
Keep calm, step by step We just got Cadog up and running and you come with Quake3


Regarding the MiniGL vs. gl4es/ogles2 window resizing thingy from above:
it's unfair to say that MiniGL would do something wrong / that gl4es/ogles2 would do something better here And, as you'll see, it's not even true:

- AFAIK the original game's window is not resizable, so it's absolutely valid that the author didn't implement proper resizing.
- MiniGL, as every other GL implementation, has functions to take care. If you use MGLResizeContext and glViewport on resize, then all is fine.
- ogles2.lib doesn't need something like an explicit MGLResizeContext. That's just a different convention compared to MGL. In MiniGL you must call this function on window resize, in ogles2 it's done automatically. Both ways of doing it are equally good / bad / legal.
- ogles2.lib automatically adjusts the glViewport internally, as if you had attached the window to the context for the first time. But ...:

Actually this is incorrect behaviour and I did just remove it
It was most likely a relic from the very first versions to enforce sane values for the viewport during development / testing.
Although you apparently liked that behaviour, it's unfortunately bad, because depending on your game's code flow it could actually interphere with your own glViewport calls...

@kas1e
Regarding GLES2 extensions in general:
Extensions are what their name implies: not mandatory, no core GLES2, not required.
And they usually don't just fall out of thin air, they usually mean additional effort

Therefore I will add extensions only if all of the following conditions are true:

- somebody actively asks for them (just like you did with GL_UNSIGNED_INT for glDrawElements).
- it's a reasonable request.
- Nova supports it.
- I can do it.
- a-eon says "do it".

Regarding the GLES2 extensions string:
it's indeed just empty until now - either simply because there was nothing to add or there's no common extension string that fits the few existing addons or because I forgot to extend the string

- "no Cube Maps for example":
ogles2.lib actually contains everything to support cube-maps because it's core GLES2. However Nova doesn't support it yet, which is why glGet(GL_MAX_CUBE_MAP_TEXTURE_SIZE) will return 0 although it should not. Once Nova supports it it will work automagically in ogles2.library (unless I made a mistake while coding it blindly ).

- GL_OES_element_index_uint:
that ext-string will be inside for the next update, I forgot to add it when I added that feature for you some days ago.

- GL_EXT_frag_depth:
supported for quite a long time already, I just forgot to add that to the ext-string (actually I wasn't aware that there even existed an extension string for it ), will be inside next update.

- GL_OES_texture_float:
not supported because nobody asked for it yet, but if I'm not mistaken Nova supports it, so it's technically possible already.


Then there are some features that are only partially supported, so that I cannot really add the matching extension string, one example being:

- GL_OES_texture_npot:
while NPOT textures including all wrap-modes are supported, mipmapping of those doesn't work reliable yet (since v1.17 ogles2 will silently disable mipmapping for such textures if you request it).

Quote:
Your test progs are far far away from becoming fill-rate-bound.

Quote:
If you supplied a window pointer then ogles2 will internally detect window size changes and adjust the back-, depth-, etc-buffers accordingly. If this happens it will also automatically adjust the viewport to match the new window dimensions.

Quote:
Here I gave you some hints how to quickly come up with something that puts more pressure on MiniGL.
However, as being said: you'd probably have to put really a lot of pressure to it because your X5000 is so fast that it can crunch even a huge amount of software TCL just like that.
I wouldn't wonder too much if even Quake2 doesn't result in such a big difference as you hope for. At least not on an X5000. On a sam460 or so this will certainly be different.

@kas1e
Quote:
LOL, and my MGLReloaded crashs in fullscreen mode

For whatever reason the MGL-version (no matter if running on top of MiniGL or MGLReloaded), as well as the gl4es version, refuse to run in window mode, only spitting out the message "no double buffer" :P

Anyway, here are the my results for fullscreen (sam460ex, tutorial level 1, pretty identical game situation, after about 30 seconds (fps need some time to stabilize / settle)) :

MiniGL: 81 fps
gl4es: 81 fps

Happy?

Cheers,
Daniel

Quote:
Great
The thellier-message above contains way too less information to come to any conclusion though, it doesn't even say if it freezes or if it creates false results or if it doesn't compile, "fail" can be pretty much anything of those.
But since he's talking about a fragment shader and the "if" itself seems to work for him, it's a most likely a totally different thing.
But without full shaders, version info, test-exe, whatever, only wild speculations are possible. Maybe he doesn't even set pos correctly in the vertex-shader, unfortunately one simply cannot tell.

Quote:
Very nice progress with gl4es
113 vs 123 fps? That means that the overhead of gl4es -> ogles2 -> Nova is really low, great!
As soon as you (can) throw sth. fatter at it, MiniGL should "lose" quickly.
Although: you're testing on an X5000, right? Presumably you'd have to throw sth. real big at MiniGL before its software TCL becomes so much of a bottleneck that the netto frame-rate drops so much that gl4es is of true benefit (or in other words: an X5000 is so fast that it can probably handle 95% of all existing MiniGL games so fast that it doesn't make a true difference, at least from game-play experience)

On a system with less CPU power all this would be more interesting. SAM users with SI will certainly benefit from this much more.
Can you please send me your build, then I can measure on my sam460 here.

@Hans
Quote:
Sounds as if there's no vertex-shader-with-if test in your suite at all so far?
Okay, that explains why this one slipped through, because if-handling in vertex-shaders is apparently broken so hard that it seems to be impossible to come up with a working one.
And it also explains why your other tests pass - because they all miss the topic

As being said: just use your HelloTriangle example with the modified shader as outlined above and already prepared for you, then you have your vertex-shader-equivalent for your test suite, you don't need to waste time coming up with sth. new.

@Hans
Quote:
Remarkably similar is not identical and fragment- is not vertex-shader

Quote:
For even more simplicity and so you don't have ogles2 in the middle and no other uniform and no other client code than your own, please do the following (and probably add the result to your list of std. tests):
Take your own W3DNHelloTriangle example that ships with Nova. Don't modify anything but the ColourArray2D.vert shader.

Here's the original shader:



Just change that to:



Compile it using
glslangvalidator -G -o ColourArray2D.vert.spv ColourArray2D.vert
(which is what your makefile does), start the triangle-example -> Freeze.
Needless to say it's the "if" only. If you remove the condition-check and always do gl_Position.x += vertCol.x , then it works.

Preprepared download of this is available here.
Curious what your working test-shaders look like. Can you please put them online?

Good luck!
Daniel