What sort of exercises can I do to specifically target my lower back, that does not involved weights like dumbbells or Kettle bells, but it is still effective to give the lumbar a workout and to get the muscle stronger? (exercises that I can do at home)
Exercises that you can do at home without weights? There's a ton of yoga poses that are great for this.
Bridges are really good and you can add weight with whatever you've got around your place.
Locust pose is really hard, and good to target the lumbar spine / lower back. Cobra pose, sphinx pose, up-dog. You can google a bunch, but yeah if you want to strengthen the lower back from home, this is a great way. You can hold these poses for a long time while watching TV, etc. I had to do these after a back injury and they are really effective for strengthening the lower back, and spine health.
If your lower back is weak, or you've suffered an injury, you won't want to do repetitive motion. Static holds will be the best.