Charge a percentage for misc parts depending on labor.
Use a design tool that will automatically provide all of the individual pieces in a sub assembly.
Use pre-made cables so you don't deal with as many loose parts.
Buy mounts and other products which include all of the common mounting hardware.
Standardize 90% of your offerings and just build everything you need into the package.
Retain all of the packaging and mark it on your "use list" during the job, this can be as simple as taking a photo with your cell phone, writing it on the box you are using, or just saving the wrappers in a pile then marking them all down.

Personally, I use all of the above, but I don't sweat the small stuff on a daily basis.
Tracking small, inexpensive parts can end up costing you more time and effort than you can recover. That's why mechanics and other industry use shop fees, job fees, materials fees, etc. The accounting costs just don't make sense for everything.