GLPs apply to any situation, environment, regulatory support, and/or processes that fall within or under the requirements set forth within Title 21 Code of Federal Regulations (with respect to all predicate rules).
If your QC lab is in support of regulated processes, assays, filings (ANDAs, INDs, NDAs, PMA, BLA, 510k, etc...) then yes, the GLPs apply.
There is not enough information here to answer your question; however, if your QC lab is doing only drug production testing, GLPs would not apply. GLPs cover non-clinical [animal] research. The "Applicability" section of the GLP regs should tell you what they cover.
if you do testing for clinical samples that would make you GCP, right? But GCP says almost nothing about working in a lab. The industry trend is to follow GLPs, but since the FDA has never and will probably not look at a clinical testing lab, does that mean we can don't have to follow any regulations?
What the regulations states is that you must also follow all predicate rules. i.e. 21 CFR Part 58 is applicable if risk says so. Conduct your own risk assessments, and make the determination based on whether your GLP process impacts, 'directly' or 'indirectly' the efficacy, purity, stability, integrateability, validity, and/or any decision making which may impact either products, regulatory data and/or filing (NDA, ANDA, IND, PMA, 510k, etc…). Again, the only way to tell is to conduct a formal ‘risk assessment’ and then determine GLP applicability.
In order to understand whether or not GLP applies to your respective Laoratory, you will need to research the FDA's Code of Federal Regulations (CFR). I have a product that allows a user to perform a full text search on all FDA Code of Federal Regulations, therefore, you can search the regs by keywords and the results will give you applicable regs in a google-like fashion where your key search terms are highlighted in yellow!
I have many clients who use the product for very similar situations you described. Check it out, as I believe you will find it useful!