There are many churches and religious organisations in the Wimmera who will welcome you and your family. Christian denominations are represented in most towns in the region. You are more than welcome to go along to a service and talk to Church members about becoming involved.
There is also an Islamic Welfare Association which includes a place to worship.
The Wimmera Information Network website has a full list of churches and religious centres.