Tutorial: Bad Geometry – How to find and fix it

In this tutorial I would like to talk about the Cleanup Command and what is considered bad polygonal geometry. Generally there seems to be a whole lot of confusion as how to use the Cleanup command and how to manually approach cleanup work. If you have never used the Cleanup command I highly recommend you take a quick look at the Maya help files. Albeit its bad reputation the article on Cleanup is actually quite helpful.

Let’s take a step-by-step look at the cleanup command.
The cleanup command can be found under Mesh->Cleanup and will launch an option box that looks as follows.
http://img210.imageshack.us/img210/8...commandcv6.gif

Cleanup Effect
In this section you can control how cleanup is going to affect your geometry. Scope is going to determine which objects Cleanup is going to target. If you set it to “Apply to selected objects” Maya will only perform the Cleanup on the objects that are currently selected in your scene. If you set it to “Apply to all polygonal objects” Maya will perform a Cleanup on all polygonal objects in your scene. I usually have this set to “Apply to selected objects” as it lets me concentrate on one object at a time.
If you set Operation to “Cleanup matching polygons” Maya will try to find and automatically fix any bad geometry, when set to “Select matching polygons” Maya only highlights any components that are causing trouble. At first it might seem like a great idea to let Maya do all the work for you, but as it turns out Maya does a pretty poor job at fixing your geometry. You should (almost) always set this option to “Select matching polygons”. Cleanup might be pretty bad at doing what it’s supposed to do (Cleanup that is); however it’s a great tool to hunt down bad geometry.
I always have “Keep construction history” checked. If you turn it off Maya will delete your object’s history after the cleanup process.

Fix by Tesselation
1. 4-sided Faces
Turn this on and Maya will look for all faces on your objects that have four sides and triangulate them if you have “Cleanup matching polygons” activated. In this case it actually makes more sense to let Maya do the cleanup. Otherwise it will only highlight 4-sided faces on your object which might prove useful if you have problems counting from one to four. 4-sided Faces works very similar to “Mesh->Triangulate” the only difference being that the former will only triangulate 4-sided Faces whilst the latter will triangulate the entire object.
http://img210.imageshack.us/img210/9...edfacesae0.gif

2. Faces with more than 4 sides
If you activate this option Maya will locate all faces that are connected by more than four vertices. These types of faces are also known as n-gons. Ongoing arguments exist all over the net debating whether you should avoid n-gons or not. I am not going to get into this argument in this tutorial. However, if you are going to apply a smooth operation on your object, be aware that n-gons can cause problems in the Mesh. Generally Maya doesn’t really care if your model isn’t entirely quadrangulated and should render just fine. If you are modeling for a Game Engine you will have to remove all n-gons and eventually triangulate your model. Don’t ever trust Maya to do the Cleanup for you – it will simply poke the adjacent faces which typically creates new problems. The best way to clean up n-gons is to either create a loop split along your mesh or terminate existing loop splits. Take a look at this picture to see the difference between Mesh->Cleanup and manual Cleanup:
http://img502.imageshack.us/img502/3...cleanupuc1.gif
Read the following article if you want further information on this topic:
http://www.subdivisionmodeling.com/f...read.php?t=907

3. Concave Faces
Turn this option on and Maya will hunt down all faces where at least two edges form a concave curve (if you don’t know what concave means it stands for curved in; the opposite being convex). Concave Faces are not a big deal in Maya. You can have convex faces in your model, never bother to clean it up and will most likely get away with it. Convex Faces can be smoothed and even Booleans can be performed on them.
http://img443.imageshack.us/img443/4...oleansmnq5.gif
You can set Maya to fix concave faces for you, but all it does is triangulate the faces. You are usually better off moving the vertices until the concavity disappears or simply ignoring concave faces altogether.
http://img443.imageshack.us/img443/8...cleanupwv2.gif

4. Faces With Holes
Faces With Holes are just what they say they are and usually result from Boolean operations.
http://img503.imageshack.us/img503/6...thholesqo5.gif
Faces with holes are basically n-gons with floating vertices. They will give you terrible results when smoothed out and should always be taken care of. Theoretically you can let Maya take care of the problem, but as usual you are better off doing the cleanup yourself. The best approach to fix this manually would be to simply delete the face with the hole, then use “Append to Polygon Tool” to close the gap, and use the “Split Polygon Tool” to terminate the floating vertices.
http://img105.imageshack.us/img105/4...cleanupgi8.gif

5. Non-planar Faces
Non-planar faces are polygons with at least four vertices that do not lie on the same plane. In other words the vertices of a Non-planar face no longer form a flat surface in 3d space. Non-planar faces are not necessarily bad; in fact Maya doesn’t care at all. Objects with non-planar faces can be smoothed, rendered and all polygonal tools should still work fine. When modeling for a game engine Non-planar faces may cause problems, but luckily they can be fixed without any problems. Cleanup will find and highlight all non-planar faces and if told to cleanup it will triangulate all non-planar faces. In this case triangulation is the easiest method to fix non-planar faces since three sided polygons (also known as trigons) always form a plane in 3d space. If you are keen about keeping your model in four sided polygons (also known as quads) you will have to clean up the geometry manually. The best way to go about it is to use the orthographic views (front, side, top, etc.) as shown in the picture below.
http://img410.imageshack.us/img410/8...arfacesyi5.gif

Remove Geometry
1. Lamina Faces
Lamina Faces are faces that share the same edges (and thus the same vertices) and they are quite difficult to spot without using Cleanup. You can think of Lamina Faces as unnecessary faces that sit right on top of each other. Objects with Lamina Faces will still render fine, but render time is likely to increase as the renderer will compute Lamina Faces twice. Any smooth operation will produce strange looking results and a SubD conversion will report a Nonmanifold error. Lamina faces usually occur when you merge vertices on faces that share the same space. Surprisingly Maya Cleanup does a great job at finding and deleting Lamina Faces. You can safely turn on “Cleanup matching polygons” and let Maya do the work for you.

2. Nonmanifold Geometry
Nonmanifold Geometry is simply put a Mesh that could not exist in the real world. The Maya help files have a pretty good definition of non-manifold geometry and I urge you to look it up yourself. Nonmanifold Geometry can be a real pain in the butt as Maya refuses to convert to Subdivisions, Booleans won’t work and smooth operations can lead to strange results. There are three different types of nonmanifold Geometry (actually four since lamina faces are technically also nonmanifold):
• Three or more faces share the same edge on an object
• Two or more faces share the same vertex, yet they share no edge
• Two or more adjacent faces have opposite normal directions
http://img62.imageshack.us/img62/460...ldfacesxy7.gif
Again, you should not let Maya do the work for you, but clean they geometry up yourself. Opposite normal directions are easily spotted by turning on Display->Polygons->Face Normals. Simply select the faces and use Normals->Reverse Normals or use Normals->Conform when dealing with a lot of inverse normals. When dealing with faces that share the same vertex you have to decide for yourself whether you want to split up the faces into separate objects or if you want to use “Append to Polygon Tool” to connect the edges. Three or more faces that share the same edge can be a bit tricky. The best advice I can give you is to take a close look at your object and visualize how this object should look in the real world. Then delete the faces that should not be there.
Here are two examples of typical non-manifold errors provided by benmcl:
http://img205.imageshack.us/img205/5...xamplesvv5.gif
http://img527.imageshack.us/img527/7...xamplesem3.gif


3. Edges with Zero Length
If you have this option turned on, Maya will look for all edges that have a very short distance and remove them. Note that the tolerance distance is not actually zero and can be changed with the slider “Length tolerance”. Maya does a decent job at removing zero length edges when “Cleanup matching polygons” is selected. However I advise you to first use “Select matching polygons” in order to find out beforehand which edges Maya is going to delete. Be careful not to set the tolerance level too high, otherwise it might backfire. Take a look at this picture where edge loops are being used to create defined edges for a Mesh Smooth. I have deliberately set the tolerance level too high to show what can happen.
http://img401.imageshack.us/img401/8...olengthgq6.gif

4. Faces with Zero Geometry Area
Similar to “Edges With Zero Length” this Cleanup operation will hunt down entire faces that have zero geometry area. Again, you can use the tolerance slider to determine the threshold of what Maya considers being ZERO. Faces with zero geometry area often occur when you begin a face extrusion and forget to actually extrude. Here is a good example of unextruded extrusions:
http://img401.imageshack.us/img401/8...eometrygu4.gif
A typical beginner mistake is to extrude an entire object and forget about it. NEVER EVER extrude in object mode - you are going to create a great mess that is hard to detect. In fact Maya Cleanup will never notice your mistake. The easiest way to spot this problem is to turn on Display->Heads Up Display->Poly Count and examine the Verts counter when drag-selecting corner vertices.
http://img210.imageshack.us/img210/3...rusionskx3.gif
One way to clean this up would be to select all the vertices and merge them. This will turn the unextruded faces into Lamina Faces which Cleanup can detect and fix.


5. Faces with zero Map Area
With this option activated Maya will look for faces that have no UV Texture Space. Faces with Zero Map are quite common. As soon as your polygonal object increases in complexity they are bound to happen. Don’t ever let Maya fix this for you – it’s simply going to delete the faces and often its adjacent faces along with it which will end up destroying your mesh. The best way to fix faces with zero map area is to unwrap the UV coordinates of your object in the UV Texture Editor. This falls into the realm of texture mapping which is beyond the scope of this tutorial.

Other examples of bad geometry

1. Overlapping Geometry
Object components that partly share the same space and thus intersect each other are considered overlapping geometry. Maya does not consider this an error but you will get strange render results since you are practically dealing with non-planar Geometry. When modeling with Subdiv Proxy you might be forced to use overlapping geometry on your Low Poly cage as shown in the following picture.
http://img211.imageshack.us/img211/9...eometryya9.gif
In this case the overlapping vertices end up looking just fine and actually help to move the two holes closer to each other in the smooth proxy version. Don’t get yourself stressed out on overlapping geometry. They can be easily fixed by moving the vertices apart and sometimes they are even necessary as in the example shown above.

2. Unmerged Vertices and Open Border Edges
Unmerged vertices and open border edges are amongst the most common type of bad geometry. They usually happen when you use the “Append to Polygon Tool” and “Extrude Edge” command and forget to merge vertices afterwards. Unmerged vertices can be found by using the Poly Count Display and looking for multiple vertices in one spot. The fastest way to find open border edges is to turn on Display->Polygons->Border Edges for your objects. If you have problems seeing the border edges you can open up the attribute editor and under Mesh Component Display crank up the border width under “Display Borders”. The easiest way to fix both unmerged vertices and open border edges is to merge the overlapping vertices by using Edit Mesh->Merge Vertices.
http://img528.imageshack.us/img528/5...ndvertilg4.gif