Loading...
> >

Questions about DIM

doubledeviantdoubledeviant Posts: 1,189
edited September 2020 in The Commons
So I recently used a temporary DIM installation (install, download, uninstall) to get the rotations add-on for Mesh Grabber (which is not available via normal download).

To be honest, I was expecting a bloated monstrosity with an awful user interface - in a nutshell, modern programming (not all, of course, but far too common).

I was pleasantly surprised to find that DIM feels rather oldschool - lean and efficient with a simple interface that doesn't hide options and settings from users.

My initial impression is quite positive, and I'm considering using DIM for future downloads. But I do have some questions:

1.) Does DIM have any built-in error-checking to insure the integrity of downloads?

2.) Does DIM take full advantage of available bandwidth by default?

3.) Are there any significant differences between the zip files downloaded by DIM and the zip files for manual installation downloaded from the Product Library section of the Daz website?

4.) What were the reasons for developing Daz Central? Are there problems or downsides to DIM that caused users to want a new program?

Thanks in advance for any helpful information or experience that you can share.
Post edited by doubledeviant on

Comments

  • NorthOf45NorthOf45 Posts: 5,556

    1) Maybe. On the rarest of occasions, downloads don't match size, but sometimes they get corrupted undetectably.

    2) You can dedicate several download connections at once (simultaneous files, mine goes to 6 max, as fast as the weakest link in the chain, some people report 300Mb/s, maybe more)

    3) Absolutely no difference. In fact, if DIM can't download for some reason, you can download manually and drop it in the download folder and install with DIM after a refresh.

    4) Apparently it was aimed at new users to make the process of installing and loading your new toys "simple, clean, and easy-to-use". See the blog. Apparently, DIM was too complicated for some users, which is probably true, but there are no really good instructions on how to set up your system for anything other than the default configuration. Central is supposed to use the same files and servers, but based on the users' "comments", not ready for prime time. Don't know if it has improved, I will never touch it.

  • doubledeviantdoubledeviant Posts: 1,189
    NorthOf45 said:
    snip

    Thanks! I was hoping for some kind of error-checking - just an MD5 check if nothing else. But still, downloading from the website via Chrono has its own issues, so DIM might not be much worse. Perhaps I'll use it for my next longer session and see how things go.
  • TaozTaoz Posts: 9,979

     

    doubledeviant said:
    NorthOf45 said:
    snip

     

    Thanks! I was hoping for some kind of error-checking - just an MD5 check if nothing else. But still, downloading from the website via Chrono has its own issues, so DIM might not be much worse. Perhaps I'll use it for my next longer session and see how things go.

    WinRAR can batch check zips.  But I've downloaded over 10.000 zips with DIM and the number of files with errors (which causes install in DIM to fail, so I suppose there is some error check there) have been less than 10 I think. So it's not a big problem, unless you have a poor connection or something.   

  • nicsttnicstt Posts: 11,715
    edited September 2020
    NorthOf45 said:

    1) Maybe. On the rarest of occasions, downloads don't match size, but sometimes they get corrupted undetectably.

    2) You can dedicate several download connections at once (simultaneous files, mine goes to 6 max, as fast as the weakest link in the chain, some people report 300Mb/s, maybe more)

    3) Absolutely no difference. In fact, if DIM can't download for some reason, you can download manually and drop it in the download folder and install with DIM after a refresh.

    4) Apparently it was aimed at new users to make the process of installing and loading your new toys "simple, clean, and easy-to-use". See the blog. Apparently, DIM was too complicated for some users, which is probably true, but there are no really good instructions on how to set up your system for anything other than the default configuration. Central is supposed to use the same files and servers, but based on the users' "comments", not ready for prime time. Don't know if it has improved, I will never touch it.

    Sounds a rediculous thing really; I mean lets fix DIM because folks can't use it. I wonder if anyone suggested actually having a simple set of instructions on installation - pretty damn sure that would have been both cheaper and not garnered the illwill Daz bloody Central seems to have.

    ... And no one considered that those folks having issues with DIM are likely to have far more issues with the more complicated piece of software that is Studio (a streamline and reduced 3D package).

    Post edited by nicstt on
  • davesodaveso Posts: 7,179
    I've been using DIM since it was released. No problems. Where it might get complicated is in defying install, download, paths etc. If you're not using default path setup. I tried Daz Central which proceeded to destroy my custom install paths. Even after deleting it, DIM DID NOT WORK properly. I'm still messing and have products in places I dont want them. If I had a huge C drive I would use default, but it's only 500gig SSD. SO I INSTALL STUFF TO D drive.
  • TaozTaoz Posts: 9,979
    edited September 2020

    OK, just tried to check 9.800+ files downloaded with DIM in WinRAR, most of them installed also.  So far it's checked about 41% but already reported 23 corrupted files (older products plus, interestingly, a PostgreSQL installer). 

    So I took a couple of those alledgedly corrupted files which had not been installed yet and installed them with DIM, and they installed without errors.  So apparently the error check during install either only work in some cases (the less than 10 bad files I mentioned were based on install errors in DIM), or the file isn't corrupted anyway.     

    Then I did a compare between the files in the zips and the installed files, (CRC32 value in zip versus CRC32 value of installed files).  The program I used reported that two files (apparently two different copies of the same file) were different, while the other file had no different files. 

    I checked some other things and found that a file in the corrupted zip is different when extracted with DIM than when extracted with WinRAR.  It contains a version number so maybe it has to do with how DIM installs updated files:

    Extracted with DIM:

    {
    version
        {
        number 4.01
        }
    }

    Extracted with WinRAR:

    {
    version
     {
     number 5
     }
    }

    It's the same file that has different CRC values in the zip versus after being installed with DIM.  So apparently DIM changes the version number in that file during installation, though strangly enough to an earlier version number (5 > 4.01).  So not sure what's going on here.

     

     

     

     

    winrar_errors_daz_zips.png
    638 x 543 - 69K
    file_compare_dim_install.png
    1267 x 652 - 167K
    Post edited by Taoz on
  • fixmypcmikefixmypcmike Posts: 19,613

    DIM does do an error-check after downloading (don't recall if it's MD5 or SHA-1).

  • doubledeviantdoubledeviant Posts: 1,189
    edited September 2020
    Thank you, Taoz - your comments focused my attention on the Test Archive feature of 7zip, which I'd used surprisingly little considering how many years I've used the program.

    Testing 1157 items (all of my downloaded 3D content from Daz, Renderosity, etc plus two game files), 7zip returned 6 errors (actually 3 files, each within a different archive, if I'm reading the report correctly).

    Surprisingly, the files with reported errors aren't the files that I'd expected - the ones with consistently stuttering downloads (which I've described elsewhere).

    Instead, they're files that seemed to download without issue (IIRC). Teddy Sleepwear: Dormir, Alascanus Merman, Alidreaux.

    The reported files for Alascanus Merman and Alidreaux seem to be .duf files for tips or hints involving materials - I'm thinking these might not be actual errors but instead some kind of workaround technique that PAs use to get this information into Daz?

    But I have no idea what .lock for TS: Dormir is... ?

    Anyway, I'm getting sleepy, so it's going on the backburner while I watch some old shows and wind down. ;)
    7zip_reported_errors.png
    830 x 592 - 52K
    Post edited by doubledeviant on
  • PerttiAPerttiA Posts: 10,024
    doubledeviant said:
    Thank you, Taoz - your comments focused my attention on the Test Archive feature of 7zip, which I'd used surprisingly little considering how many years I've used the program.

     

    Testing 1157 items (all of my downloaded 3D content from Daz, Renderosity, etc plus two game files), 7zip returned 6 errors (actually 3 files, each within a different archive, if I'm reading the report correctly).

     

    Surprisingly, the files with reported errors aren't the files that I'd expected - the ones with consistently stuttering downloads (which I've described elsewhere).

     

    Instead, they're files that seemed to download without issue (IIRC). Teddy Sleepwear: Dormir, Alascanus Merman, Alidreaux.

     

    The reported files for Alascanus Merman and Alidreaux seem to be .duf files for tips or hints involving materials - I'm thinking these might not be actual errors but instead some kind of workaround technique that PAs use to get this information into Daz?

     

    But I have no idea what .lock for TS: Dormir is... ?

     

    Anyway, I'm getting sleepy, so it's going on the backburner while I watch some old shows and wind down. ;)

    One thing that occasionally pops up as an error in the zip, are zero-length files that some use as placeholders, or at least that was the explanation I was given for them.

  • NorthOf45NorthOf45 Posts: 5,556
    PerttiA said:
    doubledeviant said:
    Thank you, Taoz - your comments focused my attention on the Test Archive feature of 7zip, which I'd used surprisingly little considering how many years I've used the program.

     

    Testing 1157 items (all of my downloaded 3D content from Daz, Renderosity, etc plus two game files), 7zip returned 6 errors (actually 3 files, each within a different archive, if I'm reading the report correctly).

     

    Surprisingly, the files with reported errors aren't the files that I'd expected - the ones with consistently stuttering downloads (which I've described elsewhere).

     

    Instead, they're files that seemed to download without issue (IIRC). Teddy Sleepwear: Dormir, Alascanus Merman, Alidreaux.

     

    The reported files for Alascanus Merman and Alidreaux seem to be .duf files for tips or hints involving materials - I'm thinking these might not be actual errors but instead some kind of workaround technique that PAs use to get this information into Daz?

     

    But I have no idea what .lock for TS: Dormir is... ?

     

    Anyway, I'm getting sleepy, so it's going on the backburner while I watch some old shows and wind down. ;)

    One thing that occasionally pops up as an error in the zip, are zero-length files that some use as placeholders, or at least that was the explanation I was given for them.

    Yes, those are zero-byte files that fail the 7-zip test, but do not interfere with anything.

  • fixmypcmikefixmypcmike Posts: 19,613
    NorthOf45 said:
    PerttiA said:
    doubledeviant said:
    Thank you, Taoz - your comments focused my attention on the Test Archive feature of 7zip, which I'd used surprisingly little considering how many years I've used the program.

     

    Testing 1157 items (all of my downloaded 3D content from Daz, Renderosity, etc plus two game files), 7zip returned 6 errors (actually 3 files, each within a different archive, if I'm reading the report correctly).

     

    Surprisingly, the files with reported errors aren't the files that I'd expected - the ones with consistently stuttering downloads (which I've described elsewhere).

     

    Instead, they're files that seemed to download without issue (IIRC). Teddy Sleepwear: Dormir, Alascanus Merman, Alidreaux.

     

    The reported files for Alascanus Merman and Alidreaux seem to be .duf files for tips or hints involving materials - I'm thinking these might not be actual errors but instead some kind of workaround technique that PAs use to get this information into Daz?

     

    But I have no idea what .lock for TS: Dormir is... ?

     

    Anyway, I'm getting sleepy, so it's going on the backburner while I watch some old shows and wind down. ;)

    One thing that occasionally pops up as an error in the zip, are zero-length files that some use as placeholders, or at least that was the explanation I was given for them.

    Yes, those are zero-byte files that fail the 7-zip test, but do not interfere with anything.

    DIM can handle zero-byte files, but some extracters (7zip, built-in Windows and MacOSX extracters) fail on them, so you lose the explanatory icons.

  • OmnifluxOmniflux Posts: 389
    Taoz said:

    Then I did a compare between the files in the zips and the installed files, (CRC32 value in zip versus CRC32 value of installed files).  The program I used reported that two files (apparently two different copies of the same file) were different, while the other file had no different files. 

    I checked some other things and found that a file in the corrupted zip is different when extracted with DIM than when extracted with WinRAR.  It contains a version number so maybe it has to do with how DIM installs updated files:

    Extracted with DIM:

    {
    version
        {
        number 4.01
        }
    }

    Extracted with WinRAR:

    {
    version
     {
     number 5
     }
    }

    It's the same file that has different CRC values in the zip versuWs after being installed with DIM.  So apparently DIM changes the version number in that file during installation, though strangly enough to an earlier version number (5 > 4.01).  So not sure what's going on here.

    While it is certainly possibly that DIM is doing this, it seems very unlikely. There are two much more likely possibilities.

    1. The ZIP file was corrupt because there are two copies of the file in the archive. This can happen when software for editing zip files (for example replacing a file in an existing archive) does so incorrectly. This can create what is referred to as a "duplicate entry" and one program may keep the first instance of the file it comes across while extracting, and another may keep the last instance.
    2. The file already existed on your system because another archive contained a file with the same path. If I recall correctly, DIM does not overwrite existing files, while WinRAR probably does. If you installed everything with DIM, you can check this by looking at the file "InstallManagerFileRegister.json" in the root of the runtime directory you installed to - it records every file that already existed when extracting and keeps a counter, and decrements the counter when uninstalling.
  • TaozTaoz Posts: 9,979
    edited September 2020

     

    Omniflux said:
    Taoz said:

    Then I did a compare between the files in the zips and the installed files, (CRC32 value in zip versus CRC32 value of installed files).  The program I used reported that two files (apparently two different copies of the same file) were different, while the other file had no different files. 

    I checked some other things and found that a file in the corrupted zip is different when extracted with DIM than when extracted with WinRAR.  It contains a version number so maybe it has to do with how DIM installs updated files:

    Extracted with DIM:

    {
    version
        {
        number 4.01
        }
    }

    Extracted with WinRAR:

    {
    version
     {
     number 5
     }
    }

    It's the same file that has different CRC values in the zip versuWs after being installed with DIM.  So apparently DIM changes the version number in that file during installation, though strangly enough to an earlier version number (5 > 4.01).  So not sure what's going on here.

    While it is certainly possibly that DIM is doing this, it seems very unlikely. There are two much more likely possibilities.

    1. The ZIP file was corrupt because there are two copies of the file in the archive. This can happen when software for editing zip files (for example replacing a file in an existing archive) does so incorrectly. This can create what is referred to as a "duplicate entry" and one program may keep the first instance of the file it comes across while extracting, and another may keep the last instance.
    2. The file already existed on your system because another archive contained a file with the same path. If I recall correctly, DIM does not overwrite existing files, while WinRAR probably does. If you installed everything with DIM, you can check this by looking at the file "InstallManagerFileRegister.json" in the root of the runtime directory you installed to - it records every file that already existed when extracting and keeps a counter, and decrements the counter when uninstalling.

    You're right, it appears to be 2.  I also installed the expansion pack for the product with DIM and it appears that it contains the same files at the same location, and overwrites the existing files from the base pack with files containing an older version number (5 -> 4.01).

    But double checking it appears that DIM does overwrite existing files, or it's told to do so in some cases, depending on some parameters somewhere.  In any case I uninstalled both packs and checked that all files were removed, then I installed the base pack and checked the CRCs for the files in question, and they matched.  Then I installed the expansion pack with DIM and checked again, which revealed that the CRC for the installed files now had changed, meaning they had been overwritten.  

    I now recall that I saw something similar some time ago with Victoria 1 - installing Victoria 2 overwrites/updates some of the files in the V1 installation.  However, after uninstalling V2 these files are now missing for V1.  See screenshots.  So something isn't working correctly here.  Uninstalling V1 does not break V2 though, so at least that seems to be working correctly. 

    As for the "InstallManagerFileRegister.json" I've scanned for all .json files in my main DAZ Library but I can't find it there.  It does exist in the root of a test library on another PC though but I can see that is has an older version of DIM (1.4.0.58) installed while the PC where the file is missing has 1.4.0.67 installed, so maybe they have changed something lately?

    belle-arum-base_dim-install.png
    1560 x 610 - 131K
    belle-arum-exp_dim-install.png
    1560 x 609 - 132K
    v1_installed_dim.png
    1560 x 625 - 147K
    v1_after-v2-installed_dim.png
    1560 x 625 - 152K
    v1_after-v2-uninstalled_dim.png
    1549 x 599 - 148K
    Post edited by Taoz on
  • TaozTaoz Posts: 9,979
    edited September 2020


    OK, did a doublecheck - this time uninstalling V2 did not break V1, it preserves the (presumably) updated files installed by V2.  I believe that the reason for this is that the files I checked the first time were installed with an earlier version of DIM - if my theory that they have changed something in DIM (the now missing "InstallManagerFileRegister.json") is correct that may explain it.  With the double check I did fresh installs of both V1 and V2 with the current version of DIM, so I assume the files now were correctly registered. 

    AFAICS this may in some cases cause problems with missing files in some products installed with older versions of DIM, if you uninstall addons or related products with a newer version of DIM than they were installed with.  A reinstall of the products in question should fixed it though, it seems.

    v1_after-v2-uninstalled_dim_update.png
    1260 x 620 - 146K
    Post edited by Taoz on
  • TaozTaoz Posts: 9,979
    edited September 2020
    Fixmypcmike said:

    DIM does do an error-check after downloading (don't recall if it's MD5 or SHA-1).

    Of the product files, or the install zip only?  In any case the checksums must be stored somewhere outside of the zips, as they only use CRC32 check AFAIK.  

    Post edited by Taoz on
  • doubledeviantdoubledeviant Posts: 1,189
    Fixmypcmike said:
    NorthOf45 said:
    PerttiA said:
    doubledeviant said:
    Thank you, Taoz - your comments focused my attention on the Test Archive feature of 7zip, which I'd used surprisingly little considering how many years I've used the program.

     

    Testing 1157 items (all of my downloaded 3D content from Daz, Renderosity, etc plus two game files), 7zip returned 6 errors (actually 3 files, each within a different archive, if I'm reading the report correctly).

     

    Surprisingly, the files with reported errors aren't the files that I'd expected - the ones with consistently stuttering downloads (which I've described elsewhere).

     

    Instead, they're files that seemed to download without issue (IIRC). Teddy Sleepwear: Dormir, Alascanus Merman, Alidreaux.

     

    The reported files for Alascanus Merman and Alidreaux seem to be .duf files for tips or hints involving materials - I'm thinking these might not be actual errors but instead some kind of workaround technique that PAs use to get this information into Daz?

     

    But I have no idea what .lock for TS: Dormir is... ?

     

    Anyway, I'm getting sleepy, so it's going on the backburner while I watch some old shows and wind down. ;)

    One thing that occasionally pops up as an error in the zip, are zero-length files that some use as placeholders, or at least that was the explanation I was given for them.

    Yes, those are zero-byte files that fail the 7-zip test, but do not interfere with anything.

    DIM can handle zero-byte files, but some extracters (7zip, built-in Windows and MacOSX extracters) fail on them, so you lose the explanatory icons.

    7zip doesn't choke on them - it warns of the potential error, but the files seem to unzip correctly (at a glance at least - I'm short on time before work today and not ready to add these to my runtime yet, so I can't 100% verify - I just tried unzipping to a folder, and it seemed to work).
  • doubledeviantdoubledeviant Posts: 1,189
    So I downloaded a significant chunk of my backlog tonight using DIM. I'm pleased with the results - 19 GB of files, downloaded faster than using the Product Library, and all passed the 7zip test except for one zero-size tip file (as mentioned above - not an actual error).

    It'd be nice to be able to download templates with DIM, too, and to be able to select an entire bundle at once, but even without those features it's a very useful tool.
  • fixmypcmikefixmypcmike Posts: 19,613

    You can filter by order number to get all the bundle contents.

Sign In or Register to comment.