[bfgane]

Hi.  I'm not sure that I am posting exactly where I should be.  If I am in the wrong place, I apologise.  I am working on testing Kodi Omega to ensure that all will work for me when the final version is available.  With Nexus, all is working well.  But when I upgrade to Omega, I am having two problems that are evident in the logs:

Log for Nexus, everything working well:

Kodi Nexus Log

But when I upgrade to Omega, I am getting delays of around 2 minutes in 2 places:

Kodi Omega Log

At the line in this log, (2024-04-01 11:54:43.462 T:21852 info <general>: Loading skin file: MyVideoNav.xml, load type: KEEP_IN_MEMORY), I have asked Kodi to display the list of all of my movies (there are 4,900).  With Nexus, I get a screen display in less than 10 seconds.  On Omega, it is taking more than 2 minutes before I get any display.

And then again, with Omega, at the bottom of the log, where it says "Stopping all services", there is again a delay of at least 90 seconds before Kodi stops. I do not have this problem with Nexus.

I realise that Omega is not yet formally released, but with it being RC2, I would expect that basic tasks like this should be smoothed out.  Please feel free to move my post elsewhere if it does not belong here.

Thank you

[Karellen]

Neither of those logs are a Debug Log so not enough info in them. Try again?

I note you are using MySQL.
For the purpose of fault finding, can you use the default database instead. I am pretty sure your issue is MySQL related, so lets test that first.

You can use multiple Portable installs so your existing setup is not touched... https://kodi.wiki/view/Portable_mode
Hopefully you are using local info, so you can rebuild your large library in about 15 minutes.

[bfgane]

Hi Karellan.  Thank you for your fast response!  Yes - I am using MySQL ... actually MariaDB ... running on my Synology DS418 NAS.

As you have suggested, I installed Omega RC1 in portable mode and recreated my database as default local.  FYI - I did not mention that in addition to my 4,900 movies, I also have about 15,000 TV episodes.  So the load took a while to build the DB!

Once it was done, I was able to have a look.  And I am pretty certain that your intuition was correct!  It appears to be a SQL problem.  Running from the local database, I can display all movies in about 4 seconds.  So its not a Kodi or Omega issue.  However, I do have a problem with query efficiency accessing my database!

The way in which I wish to access movies in the skin, is via a playlist that requests movies to be sorted in order from most recently added, to least recently added.  Therefore, in order to get "most recently added", I am guessing that the query would have to join the FILES table with the MOVIES table to gain access to the "DATE ADDED" value in FILES, then sort, and display the MOVIES attributes required.  I am guessing that this might be able to be accelerated via an index.  But are you able to assist me with the syntax of what would be required for a CREATE INDEX?

Thank you!

[CrystalP]

There were a few movie and database-related changes in v21 and what you described sounds like an inefficient query.
Is it possible that you created a custom index for performance in v20? Kodi db migration to v21 would not have included custom indexes.

The database export / import functions are recommended for database transfer and are much more efficient than rescraping, in case that's what you did.

Please provide details about your playlist / however you setup the access to the movies to allow reproduction by others.

[bfgane]

Hi @CrystalP and thank you for the assistance

My skin points to a playlist to show movies in descending dateadded order.  It executes a file that I have called Movies.xsp:

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<smartplaylist type="movies">
    <name>Kodi Movies</name>
    <match>all</match>
    <order direction="descending">dateadded</order>
</smartplaylist>

As I have mentioned prior, the population of movies that I have is 4,900.  Yes - I did rescrape; but you are correct ... I should have done an export import.

And yes I did do optimisation on V20 but have not done on V21.  I will investigate recreating that.  I believe, if memory serves, I simply did a manual ANALYZE/OPTIMIZE on the major tables such as "movies", "files", "paths" and "tvshows".  I don't know the specific syntax of the query that Kodi would have used given my playlist above.  Hence I don't know if my manual optimisation would have included all of the tables needed to be indexed to avoid a scan.

[Karellen]

It executes a file that I have called Movies.xsp:

Why not use the sort option in the movie list?

[bfgane]

Now that you point that out, I see it.  I can't recall why I went the playlist route so long ago.  And there is no query performance problem when I use the direct "movies" display vs. the playlist.

Consider this request for assistance closed.  Thank you @Karellen and @CrystalP for your time.

[PatK]

Thread marked solved.

This thread carries on, so I've reverted the close request by the op.

[bfgane]

I'm sorry but it appears I have to re-open this thread.

As per the above, I was having performance troubles when I upgraded to Omega.  But here is what I have found:

I deleted the 131 database.  Then I started Kodi from my Android box (Shield).  Once the 131 DB had been recreated, I attempted to view my Movies (5000 movies).  The initial load of the "Panel" view in my skin (Quartz) took 4 minutes!  After that, performance was ok ... but 4 minutes for initial load!?  Not great.

So I deleted 131 again.  But this time, I started Kodi from my laptop PC (Windows).  Once the DB had been created, I attempted to view my Movies.  The initial load of the "Panel" view in my skin took 8 seconds!  Much better.  

So my question is ... is the upgrade code path to create the 131 video db the same regardless of platform?  Is it possible that there are optimisation steps missing from the Kodi 131 db creation if triggered from Android vs. the version of code that is triggered from Windows?

I will produce debug logs, but I just wanted to ask the theoretical question first before I invest the time to regenerate the DBs yet another time.  I have done it twice now and the difference is amazing ... Windows-generated 131 - no problem ... Android-generated 131 - slow as a dog.

[CrystalP]

The databases are supposed to be identical, especially in a remote db situation.
Sounds like a missing index when you upgrade with the Android version. Debug log of the database upgrade should help figure this out.
It might be a permission issue.

[jogal]

Android-generated 131 - slow as a dog.

But it is slow on Android only or in all others devices (Windows) because is supposed it's a shared DB.

[zebraitis]

Android-generated 131 - slow as a dog.

But it is slow on Android only or in all others devices (Windows) because is supposed it's a shared DB.

The "slow as a dog" thing is why I abandoned android as a client.  Sure, it can do a quick playback of a movie or two...  but when you (and I) do a MySQL implementation of 5000+ movies, you start eating a lot of resources.

Decent windows-based boxes are less than $200 and fit in your hand.  Why struggle with Android?  Your time is worth more than that.

Right now, the Beelink N100 box is my favorite easy client.

[CrystalP]

OP has to clarify if the comparison is with Android client and different db creation methods or between Windows and Android clients.

[bfgane]

Regarding Android criticisms above:  The performance on my Shields (Android) is generally better (my anecdotal observation) than Windows, for Kodi, when everything is working well.  I have been really happy with the snappiness of the UI on the Shields. I can't speak for any issues that might occur on other cheap Android platforms, but the Tegra X1+ in the Shield Pro 2019 does very well.

To answer @jogal, the slowness, once the database is built, affects all of the platforms that I have, which includes a Windows 11 laptop, a Windows 10 desktop, and three Nvidia Shields.  The Shields are connected to network via hardwired ethernet.  The database is resident on a Synology DS418 NAS running MariaDB 10.11.  

Yes @CrystalP, I suspected an index problem but I tried adding various indices to solve the problem and I could not figure out which one, if that is indeed the problem.  I have also ensured that I have all of the recommended optimisations for MariaDB in place.  eg.  optimizer_search_depth = 0, query_cache_type = 1, query_cache_limit = 1MB, query_cache_size = 15MB, skip-name-resolve.

With respect to an earlier question from @Karellen, my desired order when I present movies in the skin view, is to show the most recently added movies first, then the rest in descending order.  Initially, I did try simply setting the skin option for sort by "dateAdded".  This works, but I found that Kodi would load the posters from the bottom up.  So you would end up, eventually, with the correct set of movies in the order that you want, but it would take some time for the tiles to populate up at the top, where you are viewing at first!  What I found is that if I use a smart playlist, sorted using "dateAdded", the top posters would be presented first ... so you get the desired effect with the screen fully populated (possibly after just a few seconds delay) with posters for the mosts recently added movies.  It might take 20 seconds for the full list to populate, but by the time I have finished reading the descriptions for the movies at the top, 20 seconds will have passed, so I don't notice the delay.  I don't know why this works, but it does!  As for the question at hand in this post ... I have tried both approaches (playlist on, playlist off) and it still takes up to 4 minutes to see a display.

Ok.  It sounds like I'm not going to escape without generating debug logs.  I will do so.  Please give me a couple of days!  Thank you all for your help!  I enjoy using Kodi and the support team (all of you) are a big part of the difference.

[bfgane]

Ok.  Detailed debug logs.

In the first case, I deleted MyVideos131.db.  Then I entered Kodi Omega (on a Windows machine) and caused it to recreate the 131 DB.  The debug log for this is here: debug1.  After Kodi had finished creating the DB, I attempted to enter the "Movies" panel to list the posters.  It took 4 minutes or so before I saw even the first poster displayed.  Bad.

To verify that this "badness" was not limited to just the original machine on which I built the DB above, I moved over to one of my Nvidia Shields, also running Omega.  I attempted to enter the "Movies" panel as per the above.  The log of this attempt is here: debug2.  It also took about 4 minutes before I saw even the first poster displayed.  Bad!

Back to the Windows platform.  I deleted the MyVideos131.db again.  Then I entered Kodi Omega again.  This is the same Windows machine above that had just created a slow DB.  Kodi again recreates the MyVideos131.db.  The log for this effort is here: debug3  I attempted to enter "Movies" panel, again as per the above.  This time ... success ... the first poster display came up in about 20 seconds, and all posters were populated in about 30 - 40 seconds.  For me, this is good!

And then, to test this further, I again went over to the Nvidia Shield and attempted to login to Kodi Omega again on that machine.  Success again!  The posters were fully populated on the first panel in a little less than 20 seconds.  Log is here: debug4

This is a little different than the scenario that I described above where I thought that it was Android that was building a poorly performing DB and Windows was creating a good build.  Here, it seems that it alternates.  One build is bad.  Next build is good.  But what is different?  Why is it that some of my MyVideos131.db instances perform well and others do not?

Thank you again for your assistance.  I am mystified by this one.