Yes, birth cerificates are free. You get a birth certificate shortly after birth. If for some reason lose your firs one you will have to file for a new one.
Getting a printed certificate with a seal does cost money even for the first one. The cost depends on the state in which you live. Mine (UT) is $18 for the first and $8 for each additional in the same order.