Smothing problem with modified mesh.

magne.moe_2e4aaef724

I ran into an weird problem with an dress mesh I had modified for a personal project to fit an catgirl with digtigrade legs. http://i.imgur.com/4cqC6b2.jpg

Export dress mesh to hexagon, adjust a few things and remove some frill, back to studio and use transfer utility to fit new dress to genesis, everything looks well except an small mesh bug at the hip.
Smothing should solve this I thought but smoothing tend to be the problem, picture is at smoothing 6.

Tried again this time export to hexagon, do an small change back to studio and transfer utility to fit orginal dress. same problem at hip even if hip was not touched.
The orginal dress handles high smothing well so it looks to me like its some sort of setting/ map for smothing who was lost during the transformation but I can not find it.
This loincloth shows this well http://i.imgur.com/9mEZicQ.jpg, bottom is rounded because of smothing 20 versus 3.

What is I missing here? any tips / links? tried looking at both the dress and loincloth and can not find anything.

Dress is https://www.wilmapsdigitalcreations.co.uk/view_product.php?id=145 and loincloth is from https://www.wilmapsdigitalcreations.co.uk/view_product.php?id=35

mjc1016

Both problems don't look like smoothing problems to me. 

The dress looks like there is a misplaced/improperly assigned vertex, so when you remove the frill it leaves a stray.

The loincloth looks more like what happens when Subdivision is enabled without making sure 'sharp edges and corners' is set for the item.  It does not have 'native' edge splits/bevels to control the sharpness of the edges when subdivided, so they will round over (square tend toward circular when subdivided without control loops).

magne.moe_2e4aaef724

Thank you a lot :)
Issue was that the orginal dress had smoothing type set to generic while the converted one got base shape matching as default. Changing to generic solved it.
The loincloth is set to base shape matching, changing it to generic make it square. Cloth is just one layer, here I think the rounding is an intentional nice effect.

The exploding vertexes are the bottom edge of the fabric below the buttons who end loose and into other stuff.
posible to fix but other issues with mesh makes generic better suited for the dress.