A word of caution:
With 'wild' swimming, there is always the risk of snakes in or near the water and any water that is standing with lots of leafy matter on the bottom can house leeches. Also, there is a chance of broken glass in any area, so always wear thick-soled shoes if wading.

Snakes will avoid you at all costs, leeches can be removed with salt or just flick them off, they really are not huge like in the movies.  In years of swimming outdoors, I have only come in contact with leeches 3 times.

If you get cut in any water, get out immediately and rinse the cut with clean water. Pack the wound with Neosporin and bandage. When you get home, soak it in warm water with lots of salt added, rinse afterward and use Neosporin liberally.

Use common sense and proper caution. Wild swims are wonderful fun, so much more to do than in a concrete pool; dams to build, slides to try, fish to corral, tubing along watching overhanging branches filter the summer sky above. A first whoop on a rope swing, a leap from a rocky ledge, swimming to the other side, skipping rocks, doing handstands underwater...wearing a mask and diving to see the bottom or chasing fish around, watching from the shore as a turtle slips in and swims away, being scared silly for a moment by a beaver slapping his tail. Sharing half a soggy sandwich with the greedy little fish in the shallows, scooting along a creek and seeing the biggest crawdad ever scuttle under a rock a few feet away, these are all memories that sustain a childhood.

Swann Covered Bridge
near Oneonta
park on the far side of the bridge and walk in either direction, upstream has a small sandy beach and a rope swing on the other side of the water and a good swimming hole with an excellent put-in point for tubing.
Downstream, there is a rope swing accessed by a fallen tree and it has faster water.

Caney Falls
Bankhead Forest
near Wren
¾ mile hike to the pool beneath the falls, very nice small and not deep, good for younger kids who can manage the hike, can jump through the falls from a small ledge.

Kinlock Falls
Bankhead Forest
near Wren
no hike, just steep downhill from parking area.
Proximity to main road can make trash a problem here.
Large deep pool below falls, several plunge pools above falls.
Rope to climb falls particular fun for daredevils.
Not recommended for poor swimmers-it is deep here-and the climb back to parking area is steep

Smith Lake
Various access points:
Public access easiest from day-use/campgrounds
$3 per car fee at each
Houston:
Small beach, very low use, clean bathrooms, picnic area, shallow water
Corinth:
Larger beach, clean bathrooms, higher usage level, down steep paved hill from parking lot, drop-off/pick-up area next to beach, picnic tables
Clear Creek:
Large beach, high usage, U shaped swim area makes shallow swim area, good for younger kids, clean bathrooms, picnic tables
Patrolled often by rangers.

Sipsey Picnic Area/Borden Creek
near Wren
$3 per car parking fee
short walk to Borden Creek, picnic tables, bathrooms, no showers
Shallow, good tube spot, trail follows creek upstream and downstream for several miles. Nice for younger kids, many sand banks and wading spots.

Hurricane Creek
near Cullman off 31
$3 per person fee or yearly pass available
$2 each for group of 10 or more
Highly recommend yearly pass
Gated entrance.
Small swimming hole at picnic area, plenty of wading and creek walking up and down the length of the park.
Steep climb out, bathroom, picnic tables, trash cans, very clean.
Of all the areas, I would not hesitate to go here alone with the kids.

Turkey Creek
near Pinson
Great spot! CLEAN water, home of endangered snail darter fish, waterfall you can slide down, shallow swimming hole below falls, great for younger kids, very shallow pool above falls, safe to play in without danger of being swept over the cascades. Good tubing here.
No bathrooms, no picnic area-though you can eat on the rocks near the water, wonky parking, some trash due to proximity to road, though the installation of a gate at night has cut down on this a great deal.
Patrolled by local police often, many families around.

High Falls Park
near Guntersville
Bathrooms, picnic area, playground, small creek, wide shallow swimming above falls, rope swings, deeper swimming below falls, steep climb out.

Buck's Pocket State Park
near Guntersville
bathrooms, picnic area, playground, hiking, camping available
deep, swift creek runs through park. Excellent swimming below the bridge that crosses the creek.

Canyon Mouth Park
$3 per car parking fee (summer only)
$10 for annual parking pass
Little River Canyon
near Ft. Payne
Excellent swimming hole, rope swing, trail up to second swimming hole, GREAT tubing for younger kids, swift stream connects 3 swimming areas, makes for a long chute and loads of fun.  Big rocks to jump off into deep swimming holes.
Staffed by Park Ranger in summer months
Bathrooms, picnic tables, trash cans, recycle center, water spigot and outside shower for rinsing off.

DeSoto State Park
near Ft. Payne
Above the dam at DeSoto Falls
bathrooms, picnic area, waterfall.
You can swim below the falls, but the trail is steep and not safe.
Above the dam is a nice lake, clear water, no chance of getting swept over in normal conditions, a boat ramp makes a good wading spot for little ones, bigger swimmers can walk out on the wide dam wall and jump into the deeper water.

Little River Falls
Little River Canyon
near Ft. Payne
Above the falls there is a parking area, bathrooms, picnic tables, wide paved trail to falls overlook.
You can swim and play in the water above the falls, just use good sense and caution. People have fallen to their deaths here. That being said, I have spent the day watching my 5 year old snorkel in the pools above the falls. Just stay out of the main current-not difficult-and keep to the many wide pools that are shallow and almost totally blocked off from the actual falls with rock. People that drown do so by leaping or falling off the falls into the pool below.

You can get to the pool below the falls more safely by parking at the first overlook on the Canyon Rim Parkway and hiking down to the river from there.

You can access Blue Hole, a local favorite swimming hole, from the Falls parking lot, go east on 35 to the first dirt road on the left. This has a rope swing-with a drop of about 60 feet when you let go.  I have memories of an acquaintance breaking his front tooth hitting the water one summer.   Swim above or below the falls, busy on weekends during the summer. Steep trails back from lower pool, no facilities at all.

Little River Canyon
Eberhart Point
along Canyon Rim Parkway
near Ft. Payne
Marked parking area, bathrooms, picnic tables, overlooks, steep ¾ mile trail.
Park and hike to the river, you will be one of very few who choose to do so. Swim all day at your leisure, walk up and down the river and enjoy the clean water and nearly total solitude. The walk up is very steep, but worth it.

Cheaha State Park
Cheaha Lake
$1 per person entry fee
playground, bathrooms, picnic area, fishing, paddle boats ($), hiking, campground at lake and on the mountain
Nice sandy beach, diving platform floating out in the lake, clean water, nice shallows for smaller kids.

Devil's Den Falls
near Cheaha State Park
Park at picnic area at Chinnabee Lake ($3 day use fee)
Walk ½ mile up the Chinnabee Silent Trail to Devil's Den Falls
Cheaha Falls are just past Devil's Den Falls, not far. 10 foot falls in a series of cascades, good clean swimming hole. Can be crowded summer weekends.

A word about Guntersville State Park, Joe Wheeler State Park and Fox Mallard Creek Campground beach:
The beaches are grainy, dirty and often covered in debris like trash and sticks that have washed up. The water is not clear and there are dead fish on the beach and in the water. I like fresh, clean water that is cool and clear, so I am not putting these in here as personal recommendations, there are better places to swim in Alabama.