Trying to make a 3d library database
Sfariah D
Posts: 26,519
in The Commons
I have started to work on a 3D Libary Database. I was wondering if anyone would like to help out. I am doing this for the fun of it and also to catalog my stuff. But it can be also used for other stuff later on too.
Comments
Not sure if I would have enough time to help out but I have also been thinking about looking into something like this. I hadn't got very far in my thinking yet but trying to find a solution to poses with useless names is a concern - I have bought many pose packs and now it's so tough to find anything since so many seem to have names that are more like product codes than descriptive terms of what the pose actually represents :(
Also, For figures/characters it would be nice to maybe "tag" them with keywords to help find the right one when you need a certain look for a scene.
If you can help a little bit and if you want to, you can pm me.
I have multiple versions of databases in progress for Daz assets; these are in Access. I've designed them specifically for my tracking needs so are probably not directly what anyone else needs, but I can share and help if useful.
That is cool. I forgot access was a database thing too.
I will talk more about this later, but I have started a discord server https://discord.gg/CkeeV5yB
vague much?
True, I just got on my computer for the first time today. I am still trying to figure out the project. It started out trying to get interest. I will be putting out more information soon. I am trying to figure out how to word it.
The project is started out as a way to document what products I have for Renderosity since they don't have any easy way to download all files including merchant resources, Daz Studio files, photoshop brushes and other non Poser items.
Ooops I forgot I was writing this and haven't posted it yet. I will get more info later.
I coiuld be way off base here, but if CMS is indeed a PostgreSQl database, should it not be possible to import its data into Excel or Access via an MS query data connection?
Technically that should be possible - as long as you are careful not to change anything. I saw an old thread somewhere where someone posted the outside connection details for the DB but when I tried a few weeks back it didn't work. Not sure if that means that the connection details have changed since that post or I was just doing something wrong.
Here's one somewhat recent post I see on the subject :
PostgreSQL question *SOLVED!!!* - Daz 3D Forums
I haven't done a lot of searching on the topic. For my purposes, pulling the data would save a little time in terms of data entry, but the main goal of my cataloging project is a bit different than what is likely lurking in there. I'll try to post something later - it's been a long few days IRL (which actually does involve SQL).
Yeah, that looks like the thread I saw before. I couldn't make the connection work but it was probably something I did wrong being much more familiar with MySQL over PostgresSQL. But, like yourself, my brain is also a bit fried after a week of work (only 20% or so SQL-based in my case) so will give it another go tomorrow.
I agree that this could be a starting point but ultimately cannot do all the work needed as you'd probably also need to pull from other sources, crowd-source and/or utilise some sort of AI image-recognition to really flesh out such a database.
Basically, the specific need that led me to branch out from basic cataloging of my 3D content into creating separate databases for it is this: I am working on a huge, complicated story that involves many characters and settings across time, and I was spending a ton of time looking for clothing in particular. Most of the settings of this story are not places where people wear clothes that you generally see in the Daz store, so I'm having to do a lot of improvisation. I've bought tons and tons of stuff because I "might" because able to do something with it, or some piece of a set, but now I can't find anything other than by scrolling through the Wardrobe folder.
So in short: "I need a [thing] that [fits some criteria specific to my needs]" Even if I literally pull the data out of the Daz database, it can only go so far. So I created this ugly hodgepodge, which was never meant to be seen by anyone other than me:
Essentially, what I've done here is create a structure where I have an asset (the actual thing acquired from Daz or elsewhere), which has one or more child records (each individual part of that thing). Each child record has a type (coats, shirts, etc) and some various notes, as well as a type-specific set of attributes that could be relevant. I also have a screen shot that clearly shows each child object. This jacket, for example, has such significant length morphs that it counts as a long or short jacket. I'm constantly adding new attributes as I go along and realize there's something that needs to be added. This sets up being able to query for a thing that meets a set of criteria: "I want a long coat" or "I want a long, loose skirt."
This database also includes hair and accessories.
Obviously the details and attributes that are important to me are not what anyone else would care about, but this general approach seems to be accomplishing the need.
Edited to add: I had created a similar DB for poses in the past, which - if I get time - I'll update to be similar to this one. Daz's breakdown for function is helpful, but like here, I had done a much more detailed set of attributes.
Yeah, I understand that. I think many of us tend to buy stuff that we do not need immediately but may be useful in future endeavours. This does lead to forgetting the things you have and so inviting the need for something to help. Smart Content is a step in this direction but rendered pretty unreliable due to some PAs not filling it out properly and also it not quite going far enough. Like you have said sometimes you want a long coat and there is no good way to search for that. I guess the relevant Smart Content category for that would be Wardrobe/Outerwear/Tops - for me I only have 218 in there to look through but still have to scroll past capes, aprons and for some silly reason a bra and shorts (sigh). Obviously, this would be worse for people with a larger amount of coats and general outfits than I have. For me the major issues I have are with poses where I have many Smart Content categories with over the 1,200 count of items that would be awkward to scroll through since there is no other way to narrow them down to something. So I would like a way to "tag" them in a similar way to what you have done with your attributes. I guess it would be possible to add custom categories to items but you would still be working within a tree structure and so would not give you the flexibility to search by multiple categories - e.g. "Female + Standing + In Heels + Hands on Hips".
I wouldn't put your database down like that. At the end of the day it's just a tool to help you achieve the project you actually want to make and so it doesn't need to be pretty or impress anyone else, it just needs to work for you. If it does that then you can be proud of it :)
Just seen your edit - I would be interested to see what you did there. Was it all manually keyed in by yourself or did you try and parse the dsf to try to automatically figure out what it was doing and apply attributes that way?
It was all created manually, aside from copying/pasting the name of the product.
The pose database is older and less flexible, and never saw more than test use. I was accommodating for different possibilities by creating multiple child records for each pose, rather than what I did here, where I have the type-specific attribute list. Thus, just one attribute record that has Yes/No checkboxes for everything relevant. If/when I redo it, it'll be redone in this style, with the pop-up attributes specific to each pose type and a standard screen shot for each pose. My approach to poses had been to start with a pose (eg, standing) and then add interaction attributes (is this pose something that is normally related to interacting with others, eg greeting, combat, etc) and what, if anything, is the emotion of the pose (if you saw someone in this position, do you read hostility? Shyness? etc). So if I wanted to consider some standing poses for an annoyed person, instead of scrolling through a bazilion standing poses, I could narrow down my list.
I admit that this betrays my past, in which I have spent a lot of time doing detailed data entry ... and I like to do it. Unfortunately, the practical offset is that one only has so many minutes in a day (a week, a year, a life ...) and at some point, I have to apportion out how many of those minutes are spent analyzing poses for a database vs creating art with them vs writing the damn book that all of this was ostensibly for.
Given that we have kept talking about this despite trying to put it off until later I have also taken another run at this. Turns out it was 100% down to me not understanding Postgres and so entering the connection details slightly wrong. I now have a DB client connected and can see all the juicy raw data :D
If you'd care to share the nuts 'n bolts, I could see some value in pulling what Daz has to say, and elaborating from there.
I'm in the data business in large part because I love juicy raw data. :)
It seems we have much in common. One of the reasons I have been hesitant to go down this path is that I know that if I did build something and allow the possibility to expand the keyword tagging for my items then I would be compelled to fill it in for all assets - and while I don't have a library as massive as some people here that would still be a huge undertaking. I wouldn't say that I enjoy data-entry itself but I do like having the data and for it to be correct and complete (therefore the items in the "Lost and Found" category are still somewhat of an annoyance to me).
Perhaps I will play around with some stuff in the coming weeks, depending how much time I can spare - I still want to be working on some actual renders afterall :)
Oh, well the connection details were mainly in the forum thread you posted - except the table name which I misread from the image. The issue happened because I normally use MySQLWorkbench but that seemed the wrong tool to connect to a non-MySQL DB and so I tried out Beekeeper which had been on my list to look into for a little while. However, for some reason Beekeeper cannot connect without a table name and so I was trying to use dzcontent as that looked like the correct value to me. Turns out the table name to use is just "Content" which is probably completely obvious to someone more familiar with Postgres but I just didn't see it before - mainly because I stopped looking as at the time I was just curious and not actually really caring about doing anything. I have attached a screenshot of the settings.
I wouldn't bother using pgAdmin (unless you grab an old version) since it constantly complains about the "wait_event_type" column being missing - this is due to the Daz CMS using an old version of Postgres. I couldn't see a way to either tell pgAdmin to stop checking or configure the client to use the old column name.
Here's the thing, though: What if you made this database thingy a distributable product, with a means of letting other people add their own tags and stuff to the products they have, let the script query a central server somewhere that lets them upload those as they accumulate, lets the script detect if someone else has placed items you have onto the server and grab the details into itself from the server... this could become a crowd-sourced, build-a-better, searchable-DB project.
24,000 posts on this forum. wondering if theyre all like this...
There is a product database for Renderosity here, it's been broken for a while since they put captcha on their login, but I've found another way to get in so it's actually working again now, just need to fix some minor things. It's a commercial project but free for up to 500 products.
https://www.taosoft.dk/checkout/pm-rr/index.htm
Not really. I was going to post more but it seemed that the thread gotten taken over. I am getting distracted by different things which is why the delay.
oh ok