DIM startup time [solved]

My DIM startup time, from the moment of logging in with my details, to the moment of seeing the product updates and my new purchases is over 30 minutes.
Is this normal? :-o
My content is on an external USB 3 drive, drive speed is about the same as the internal drive, so I'm assuming that it shouldn't make a difference.
Post edited by Ati on
Comments
That does sound extreme, mine takes a while but more like ten minutes I'd think - and I have somethign like a third of the store judging by the Showing X of Y items.
I get about the same as Ati, or more. Same hardware, only Windows updates. Makes me hesitate to refresh or restart too often. I do have many products, but the time seems to be increasing exponentially (as in: twice as much stuff takes 4 times longer). I can see the disk activity as it scans the manifest and package archive folders. DIM has been updated a few times over the last couple of years, so maybe all the added features have increased the overhead.
Seriously folks?
My content is also on an external drive. I have 4594 products installed. I just tested my starup to display time. It was 17.1 seconds. Win 7 64 bit, 32GB Ram, I7-4770K.
Mine is on an internal SSD, and it takes maybe 2-3 minutes.
My system is almost identical: i7-4770k, 32GB RAM, the only difference is that it's a win 8.1. I have more products, but based on the "how many pages in your product library" thread, I'm definitely not the one with the most items.
@NorthOf45's post encouraged me a little that at least I'm not alone, but the rest of your posts didn't. :D
What are the tricks to speed this up?
@Ati
I think the biggest issue is your internet connection to DAZ (and their response). DIM is occasionally slower to open, but never more than a minute. i timed my opening several times yesterday and all times were similar.
13 seconds today. No tricks that I know of. This has always been typical for me.
I'm on a gigabit line. Once DIM starts, download speeds can be up to and over 500 Mbps.
Are you sure the drive is connected to a USB 3 port on the computer? I have the same setup, on a laptop, with over 13,000 items showing in DIM and my startup is closer to 10 minutes than 30. I'll time it later today when I'm at my wifi hotspot.
Yes. It's in the blue port, and now I also checked it with UsbTreeView, to make sure, and it says:
I don't think it's the internet connection, I see very little activity while DIM is starting. What I do see is a lot of disk activity (one for downloads, another for the library itself), presumably the download manifests being replaced (and I keep the product archives) and the installation manifests being scanned to sort out what's been installed. I did move the installation manifest archive to the library disk with the last DIM update, so maybe that's slowing things down, but it's not trying to read at full speed. I can get sustained 60-80 MB/s transfers from both disks.
Just did a refresh to get some numbers. When DIM is refreshing, it's about 20MB/s on the product archives for several minutes (~15 minutes). I see a noticeable increase in activity of indexing services, that could be one source of delay. Then it switches to the installation manifests (~4 minutes), then another minute or two to crunch the numbers and display. So, 20 minutes total, a good day. I do remember longer, though. Also, watching with Resource Monitor, every file that is touched is going through the anti-malware scanner (MsMpEng.exe). I'll try again with indexing paused to see if that speeds things up.
WIN 10, R7-1700, 32 GB RAM
Hmm, could it be the virus scanner? I'll have to give it a try next time I start DIM up. Thanks for the tip!
Okay, I figured out what is causing this huge load time.
I kept all the downloads, and for some reason, DIM created new xml files for them (or just updated them) every single time it was started.
I moved all installed downloads to another directory that DIM doesn't know about, and it now starts up quickly.
Well - not solved, just worked around. Yes, DIM does at the very least update the date and time modified for all of the .dsx files. But I've got all 13,000+ products with xml in my download directory and it just took a bit less than six and a half minutes to start up and let me grab the thumbnails for the new downloads.I do NOT index the drive my download directory is on, and I have explicitly excluded the download directory from my AV - I trust Daz to not have malicious content in their downloads.
Okay, paused indexing, excluded the library and download folders from malware scans. Still took about the same time, maybe 2 minutes less, so ~18 min. I recently discovered that the archive drive (6 TB) is exFAT, which had pre-formatted sectors of 1MB. Lots of slack for those .dsx files of 1K each (99%!!). The download folder refresh takes the most time, so I am sure that has something to do with it. Maybe if it was NTFS...
I realized why my DIM opens so much faster. I recently changed settings to delete files after download.
I did this for a couple of reasons: 1) I install everything I purchase, 2) I backup my Studio Content weekly.
I decided it was just eating up space even though I have plenty of storage. Given weekly backups it was also making the backup files larger and taking longer for unecessarily redudant backup.
I may need to do that too, @fastbike1. I just had to move my download folder from D to E to spread the space. My download folder was over 650 GB. That's a LOT of storage to be using "just in case" I ever have to rebuild my content library again. And, tbh, my download speeds are fast enough it wouldn't kill me if I did have to.
And if it speeds up my DIM startup time, it may be worth the "risk".
@JonnyRay
That was my reasoning and conclusion. DIM opening time went from several minutes to the few seconds I've mentioned.
I think their 17.1 seconds is to 'login' and not to actually populate their list of 'Installed' products. That's about how long my computer takes to get to that point (7 year old 3rd gen i7).
If I include the time it takes my computer to populate the 'Installed' list of DAZ products with 4037 products it take 3 minutes and 30 seconds give or take 10 seconds.
I have HP 8470P Elitebook i7 16GB RAM, with my DAZ Content stored on an internal Sata III 2TB drive at 5400rpm with 64MB (or is it 64GB???) SSD cache (Seagate Firecuda drive or something like that).
I blew my download folder away and told DIM to delete after istalling. I need that space for other things.
@nonesuch00 "I think their 17.1 seconds is to 'login' and not to actually populate their list of 'Installed' products"
That's not correct. That timing for me is when I see any new products/updates, i.e when DIM is ready for me to take action.