It is illegal to release any fish into public waters unless caught from that water.  If you catch your own bait, it may only be used in the water from where it was taken.