FDA下一阶段更加关注OOS调查   FDA药品审评与研究办公室（CDER）国际药品质量处处长CarmeloRosa于去年11月4-7日在费城召开的ISPE年会上表示，随着FDA对制药行业数据可靠性检查的结果开始显现，在其中发现了更多关于超标（OOS）调查的问题。   一些公司不再掩盖分析实验室的超标结果，使得最容易出问题之处显露出来。Rosa表示：“现在我们要做的是调查，但是调查本身却有缺陷。”多位行业顾问表示，OOS调查使他们非常忙碌。一个原因在于，正如Rosa所说的，由于已经加强了数据可靠性，一些客户必须进行更多的此类调查，或许还要学习如何开展这些调查。 但问题同样也出现在数据可靠性方面被视为强项的大型制药公司。虽然可能存在人员流动率与员工胜任程度等方面的问题，但目前尚不清楚具体原因。有猜测认为，一些制药商可能会将这些OOS调查的责任推到企业内较低层级员工身上，这些层级的员工可能是不具备高学历的工程师或技工，他们缺乏找到根本原因所需的分析架构。   尺有所短：表现强劲的工厂也有差劲的实验室 2016年，瑞士St. Gallen大学在开展FDA资助的一项研究发现OOS调查问题的一些线索。 领导这一研究项目的Thomas Friedli教授2018年11月7日在费城ISPE年会上所作的报告中，讨论了一个重要研究结果：就质量来讲，制药设施与其分析实验室并不总是处于同一条轨道上。Friedli表示，在差劲的工厂中可以有非常好的实验室，而在表现强劲的工厂中，也会有差劲的实验室。 OOS调查相关缺陷汇总 频繁将OOS归因为人为因素   The reportspoke of hundreds of cases in which OOS-results had been invalidated withoutscientific and traceable root-cause analysis because of human errors. It wasfurther considered a critical deficiency that the deviation and OOS management(including related protocol and review systems) was designed in a way so thatdiscrepancies were systematically not documented. 缺陷报告提到调查归因为人为失误的OOS结果达数百个之多，而没有科学和可追溯的根本原因分析。检查人员认为该公司的偏差和OOS管理（包括相关方案和回顾体系）未能有效记录异常，并进一步确定为严重缺陷。   将OOS结果归因为人员不遵守SOP，但未解释人员为何不遵守SOP   Your firm failed to thoroughly investigateout-of-specification (OOS) assay test results for your over-the-counter (b)(4)cream, lots (b)(4),(b)(4), and (b)(4). You retested the samples and invalidatedthe OOS results without any scientific justification.You released these lotsinto the U.S. market in September 2017. 你公司未能彻底调查你们的OTC药品XX膏批号XX、XX和XX的OOS含量检测结果。你们复测了样品，然后宣布OOS结果无效，却并无任何科学论证。你们将这些批次于2017年9月放行到了美国市场。   In your response, you stated that your analystsfailed tofollow yourOOS checklist procedures. You committed to revising theapplicable procedures and retraining analysts. 在你们的回复中，你们声称你们的化验员未能遵守你们的OOS检查清单程序。你们承诺会修订适用的程序并重新培训化验员。   Your response is inadequate because you did notextend thescope of your investigation to include a retrospective evaluation ofall OOS results that were invalidated without scientific justification. You didnot determine the root cause for the original OOS results or explain why youranalysts failed to follow your procedure. 你们的回复是不充分的，因为你们并未延伸你们的调查，包括回顾性评估所有被宣布无效而没有科学论证的OOS结果。你们并未确定原始OOS结果的根本原因，亦未解释为何你们化验员未能遵守你们的程序。   In response to this letter, provide a summaryreport on the retrospective review of all invalidated OOS results obtained forproductsdistributed to the U.S. market. If your investigation reveals that youreleased drug products that did not meet specifications, indicate thecorrective actionsyou have taken or will take, such as notifying customers orrecalling products. 在回复此函时，请提交一份回顾所有销售至美国市场产品被宣布无效的OOS结果的综述报告。如果你们的调查显示出你们放行的药品并不符合质量标准，请写明你们已采取和将要采取的纠正措施，如通知客户或召回药品。   ——韩国Kolmar Korea Co.Ltd.   OOS归因为人员因素（实验员错误）后，未对该实验员所做所有实验进行回顾 Your OOS investigation numbers 310016652/310016653 for (b)(4) Tablets (b)(4)mg batches (b)(4)/ (b)(4) initiated January 7, 2019 and closed (b)(4), identified the root cause for OOS assay results as analyst used the incorrect diluent. You did not thoroughly evaluate other tests performed by this analyst by verifying the accuracy of results reported by this analyst. Your CAPA plan did not include verification of the accuracy of other results reported by this analyst. You concluded in your Impact Assessment you “Verified the concern analyst previous analysis and no OOS/OOT results derived due to analytical error. Hence there is no impact on the previous analysis of concern analyst”. Your verification consisted of confirming this analyst was not associated with other OOS/OOT investigations. Alltests and results performed by this analyst are suspect until you verify the accuracy of the results reported by this analyst. You do not have data showing you tested and confirmed samples tested by this analyst are within the known variance of the test procedure and that the results reported by this analystare reliable. 你们的OOS调查编号310016652/310016653，是关于某片剂含量OOS，启动日期20190107，关闭日期XX，识别根本原因为化验员使用了错误的稀释剂。你们没有通过验证该化验员所报告结果的准确性彻底评估由该化验员执行的其它检查。你们的CAPA计划不包括对该化验员报告的其它结果的准确性的验证。你们的影响性评估得出结论说你们“验证了该化验员之前的分析，没有因为分析错误发生的OOS/OOT结果。因此对该化验员之前的分析没有影响”。你们的验证只是证明了该化验员与其它OOS/OOT调查没有关联。如果你们没有验证该化验员报告的结果的准确性，所有由该化验员执行的检测和结果均是可疑的。你们没有数据证明你们对该化验员所检测的样品进行了检测并确认其结果在分析方法的已知波动范围内，证明该化验员所报告的结果是可靠的。 OOS归因后，评估不充分 Your OOS 310016465 documents out of specification assay results for (b)(4) Capsules USP (b)(4)mg batch (b)(4) during six-month long-term stability study conditions. You have been utilizing an in-house test method which you validated February 2008. That method requires use of (b)(4) capsules to be dissolved in (b)(4)ml of solution (b)(4) micrograms/ml. You performed hypothesis testing as part of this OOS investigation and changed the method to utilize (b)(4) capsules dissolved in (b)(4) ml of solution((b)(4) micrograms/ml). Results obtained from this hypothesis testing,utilizing a different concentration from your original method, were within specification. Rather than testing the other (b)(4) batches on the U.S. market, you conducted an impact assessment. You do not have a scientifically valid reason for recalculating the results of the previously tested, approved and shipped batches. You have not verified your original in-house test method nor the un-validated method you used to conduct analyses for (b)(4)mg strength (b)(4) Capsules USP are at least equivalent to the USP monograph method. Additionally, there are currently no less than (b)(4) batches of the (b)(4)mg strength of (b)(4) Capsules USP on the U.S. Market and you have not performed the comparison test between the USP monograph method and your in-house method utilized to test,approve and ship batches of that strength of the drug product. 你们的OOS 310016465记录了某胶囊6个月长期稳定性试验中含量OOS。你们使用的是2008年2月验证过的内控方法。该方法要使用XX粒胶囊溶于XXML的XX微生物/ML的溶液中。你们进行了假设性测试，作为此OOS调查的一部分，并修改了方法将XX粒胶囊溶于XXML的XX微生物/ML的溶液中。该方法修订了原始方法中的浓度，所得结果符合质量标准。你们没有检测美国市场上有其它XX批次，只是进行了影响性评估。你们没有科学有效理由来重新计算原来检测并批准发货的批次的结果。你们未验证你们原来的内控方法，亦未验证你们用来分析某胶囊的方法至少等同于USP各论方法。另外，目前美国市场上至少有XX批次该胶囊，你们没有将你们用于检测、批准和发货批次的内控方法与USP各论方法进行对比检测。 ——Dr.Reddy’s Laboratories Ltd OOS纠偏措施2年未实施   Your firm rejected (b)(4) batch (b)(4) for assayfailure on October 23, 2015. You opened an investigation the same day andsubsequently closed the investigation a few months later on March 20, 2016. Youdetermined the root cause of the failure to be (b)(4) from the (b)(4)in March,2016, but as of our December 2017 inspection, more than two years after thefailure, you had not implemented a corrective action to address the root causeyou identified. In the interim, your quality unit released at least (b)(4)batches of finished (b)(4) API. 你公司于2015年10月23日因含量不合格而拒收了XX批号。你们在同一天启动了调查，在几个月之后于2016年3月20日关闭了调查。你们在2016年3月确定不合格的根本原因是XX中的XX，但直到2017年12月检查时，在不合格发生2年之后，你们仍未实施纠正措施来解决你们所识别出的根本原因。在此期间，你们质量部门放行了至少XX批次的XX原料药成品。   Your response stated that you will reopen theinvestigation into this failure. You also stated that the (b)(4) subsequentlyreleased batches met specifications, and analytical trends were consistent withpreviously manufactured batches. However, you failed to include data supportingthis statement. 你们的回复声称你们会重新开启针对此不合格的调查。你们还声称之后放行的XX批次均符合质量标准，并且分析趋势与之前所生批次是一致的。但是你们并未包括数据来支持这一声明。   In your response to this letter, provide a detailedsummaryof your investigations. Assess the risk of your decision to manufactureand release an additional (b)(4) batches even though you had not implementedany corrective action to resolve the problem your own investigation identifiedas the root cause for batch (b)(4)’s assay failure. Include any additionalsampling and testing you performed between October 25, 2015, and December 2017to support your release decisions. 在你们对本函的回复中，请提交一份详细的调查总结。评估你们并未执行任何纠正措施来解决你们在调查中发现认为是XX含量不合格根本原因的问题，就决定生产和放行其它XX批次的风险。在其中包括所有于2015年10月25至2017年12月期间的附加取样和检测来支持你们的放行决策。   In addition, provide an action plan with timeframesfor a global assessment of your corrective actions performed over the pastthreeyears. Ensure this assessment identifies all investigations and thecorrective actions and preventive actions that you initiated in response toyour investigation findings, and that it shows how you determined that youexpanded your investigations to other potentially affected batches. 此外，请提交一份对你们在过去3年实施的纠正措施的全球评估时间表和行动计划。确保此评估能识别出所有调查和针对调查中的发现而启动的CAPA，以及显示你们如何确定你们延伸调查至其它潜在受影响批次的情况。   将微生物超标OOS归因为生物安全柜污染，但无充分的证据说明   Your quality unit released (b)(4) batch (b)(4)onApril 30, 2015, despite a total aerobic count of greater than (b)(4) colonyforming units/gram (CFU/g). The specification is less than (b)(4)CFU/g. Yousold this batch of (b)(4) to a firm that manufactures sterile finished drugproducts used to treat (b)(4) patients in the United States. 你们质量部门在2015年4月30日放行了XX批次XX产品，但该批次总需氧菌计数大于XX菌落数/克（CFU/g）。质量标准是小于XXCFU/g。你们将此批次销给了一个生产无菌制剂产品用于治疗美国XX患者的公司。   When you observed the failing result on April 15,2015, you initiated an out-of-specification (OOS) investigation. When youclosed the investigation on April 24, 2015, you concluded, without adequateevidence, that a biohazard cabinet was a potential source of contamination andreleased the batch. 在你们于2015年4月15日发现此不合格结果时，你们启动了OOS调查。当你们于20150404关闭此调查时，你们得出结论说生物安全柜是潜在污染源，然后放行了此批次，但并无充分的证据。   Your response stated that you revised your SOP forhandling OOS results to clarify resampling and retesting. However, the rootcause you identified lacked scientific justification, and you did not provideevidencethat supported your release of batch (b)(4). 你们的回复声称你们修改了你们的OOS结果处理SOP，写明重新取样和重新检测。但是，你们所发现的根本原因缺乏科学论证，你们并未提交证据支持你们放行XX批。   In your response to this letter, assess the risksof your decision to release batch (b)(4) despite the microbiological testfailure and despite the inadequacy of your investigation into the cause of thefailure. Provide your plans for addressing product quality and patient safetyrisks for any drugs still in distribution, including potential recalls ormarketwithdrawals. Provide an updated OOS procedure that requires the speciation ofmicrobes found as a result of any microbial analysis. 在你们对此函的回复中，请评估你们在微生物检测不合格情况下，以及你们对不合格原因调查不充分情况下决定放行XX的风险。提交你们解决仍在销售中所有产品的质量和患者安全风险的计划，包括可能的召回或市场撤回。提交一份更新后的OOS程序，要求将微生物形态作为所有微生物分析的结果。   频繁判定OOS结果无效   Your firm frequently invalidatedinitialout-of-specification (OOS) laboratory results without an adequateinvestigationthat addressed potential manufacturing causes. 你公司频繁地宣布初始的OOS化验室结果无效，却并无充分的调查来说明潜在的生产原因。 A.     AssayFailure 含量不合格   While conducting component release testing on (b)(4)activepharmaceutical ingredient (API) batch (b)(4), your firm obtaineda failing assayresult of (b)(4)% (specification (b)(4)% to (b)(4)%). 在检测某批次中某API组分放行测试时，你公司得到了一个不合格的含量结果为XX%（标准为YY%-ZZ%）。   Data from the investigation demonstrated thatmultipleretest results were comparable to the initial OOS result. Initialretestsyielded four results ranging from (b)(4)% to (b)(4)%. Theseresultsincluded a freshly prepared sample, which tested at (b)(4)%. 调查数据显示多次复测结果与初始的OOS结果具有可比性。首次复测得到从XX%至YY%的4个结果。这些结果包括一份新制备的样品，其检测结果为ZZ%。   A second analyst tested a new set of samplesandobtained results including (b)(4)%, (b)(4)%, and (b)(4)%.You then performedthe test again. Only the last samples yielded significantlydifferent assayresults ((b)(4)–(b)(4)%). 另一化验员测试了一系列的新样品，获得结果为XX%、YY%和ZZ%。于是你们重新进行了检测。只有最后的样品获得了有着显著差异的结果AA-BB%。   Despite the findings of multiple values close totheoriginal OOS value, your firm invalidated the initial failing result,statingthat the initial result “shall be considered an outlier and retestresults shallbe reported as final results.” Although the investigation failedto identify aconclusive laboratory root cause, you did not conduct anevaluation of yoursupplier, and reported an average result for batch release. 尽管多个结果均接近原始OOS值，但你公司仍宣布初始不合格结果无效，声明初始结果“应作为离群值，复测结果应报告为最终结果”。尽管调查并未发现结论性的化验室根本原因，你们并未对你们供应商进行评估，而是报告了平均结果用于批放行。   It is not appropriate to use an “outlier test”toinvalidate your API assay result. Such statistical treatments do not identifythecause of an extreme observation, and are only of informational use inaninvestigation of chemical testing. Further, in this case, yourinvestigationincluded multiple retests that were the same or very similar tothe originalOOS result. 使用“离群检查”来宣布你们的API含量结果无效是不恰当的。此类统计学处理并不能识别出极端值的原因，在化学检测的调查中只能用作参考。另外，在此案例中，你们的调查包括了多次与原始OOS结果相同或非常相似的结果。   Our inspection also revealed additionalinappropriateuses of outlier testing. Your firm released other raw materialsand drugproduct batches by retesting and concluding that the original OOSresult was an“outlier.” 我们的调查还发现有其它的不当使用离群检查。你公司通过复测并得出结论说原始的OOS结果为“离群值”从而放行了其它原料和药品批次。   We acknowledge your firm’s change control onJanuary18, 2017, to remove the outlier test in your Handling of OutofSpecification Test Results standard operating procedure (SOP). Youalsoreversed your original decision to release(b)(4) batch (b)(4),and have nowrejected it. However, your response did not address API qualityissues that mayhave caused the low assay, and lacked an adequate reassessmentof the otherbatches released with the outlier test. 我们知晓你们公司在2017年1月18日的变更控制中删除了你们SOP《OOS检测结果处理程序》中的离群值检查。你们还推翻了你们原来放行XX批和YY批的决定，现在将其拒收了。但是，你们的回复并未说明可能会引起低含量的API质量问题，并且缺乏对采用离群值检查旅行的其它批准所做的充分的重新评估。   B.    Content Uniformity Failure 含量均一性不合格   Your investigation of content uniformity OOSresultsfor (b)(4) mg tablets for batch (b)(4) was inadequate. Twoindividualunits and the acceptance value (AV) were OOS for this batch, whichwas anexhibit batch filed in your (b)(4). 你们对某批次XXmg片剂的含量均一性OOS结果所做的调查是不充分的。该批次的2个单剂量和可接受值（AV）均为OOS，这2个批次是你们XX申报中的批次。   The initial assessment of the OOS results foundnoevidence of laboratory error by the analyst. Retests from stock solutionandre-sonicated samples yielded results consistent with the original OOSresults,and ruled out improper sonication and dilution error as root causes.Althoughthe investigation did not demonstrate a conclusive assignable cause,yousurmised that the “probable laboratory error” was inadequate cleaning of the(b)(4)shaft by the analyst. You then invalidated the initial OOS results andreportedtest results from a new set of (b)(4) units that passedspecifications.Your firm’s investigation indicated that this “confirmed” thatthere was alaboratory error. 对OOS结果的初始评估未发现证据证明化验员引起的化验室错误。对贮备液和重新超声的样品复测所得结果与原始OOS结果一致，排除了超声和稀释错误的根本原因。尽管调查并未证明有结论性可归因原因，你们臆测“可能的化验室错误”是分析员对XX轴清洁不充分。然后你们宣布初始OOS结果无效，报告了一套新的合格的检测结果。你们公司的调查说这样就“确认”了存在化验室错误。   When an investigation lacks conclusive evidenceoflaboratory error, a thorough investigation of potential manufacturingcausesmust be performed. The test of(b)(4) units for content uniformityisintended to represent different parts of a batch, and to help detectifportions of the batch are non-uniform. Your acceptance of the resultsfromtesting a new set of (b)(4) units based on an unproven hypothesiswasinsufficient to conclude the investigation. In addition to furtherlaboratoryinvestigation, your firm should have identified possiblemanufacturing causesand more extensively characterized uniformity of the batch. 如果一份调查缺乏结论性证据证明化验室错误，则必须对潜在的生产原因进行彻底调查。对XX剂含量均一度的检测意在代表同一批次内的不同部分，帮助发现该批次是否不均一。你们基于未经证实的假设来接受一套新的检测结果是不足以对该调查做出结论的。除了深入的化验室调查以外，你们公司还应识别出该批次可能的生产原因，以及更为广泛的特性均一性。   Insufficient cleaning of the (b)(4) shaft wasalsocited in a previous investigation as the presumed root cause of OOScontentuniformity results for batch (b)(4) of(b)(4) mg tablets.A different analystperformed that test, and your corrective action andpreventive action (CAPA)plan required retraining staff. XX轴清洁不充分在之前的调查中也曾被引用作为XX批XXmg片剂假定的含量均一性OOS结果的根本原因。一个不同的化验室实施了检测，然后你们的纠正预防措施计划要求对员工进行重新培训。   In your response, you indicate that SOP CQA-063AnalystQualification has been revised to state that “if (b)(4) times errorisrepeated then the analyst shall be requalified on the specificanalyticaltechnique.” However, your response did not sufficiently addressimprovements inyour quality system that will ensure that flaws in writtenlaboratoryprocedures and analytical equipment will be corrected. 在你们的回复中，你们说SOP CQA-063“化验室资格确认”已进行了修订，声明“如果错误重复XX次，则该分析员应对某项分析技术重新进行资格确认”。但是，你们的回复中并未充分说明对你们质量体系的改进，以确保会纠正书面的化验室程序和分析设备中的缺陷。   ——Lupin Goa   无效OOS发生率过高   Your firm invalidated initial OOS laboratoryresultswithout adequate investigations. From January 1, 2015, to December 31,2016,you invalidated nearly all (134 out of 139) initial OOS results andattributedthem to laboratory error. Although some investigations failed toclearlyestablish that laboratory error occurred, you did not conduct afull-scaleinvestigation to thoroughly review potential manufacturing causes andassesscommercial history to identify similar instances of high variation or OOSresults. 你们公司未经充分调查即宣布初始OOS结果无效。自2015年1月1日开始至2016年12月31日，你们宣布了几乎所有的初始OOS结论无效（共139个，其中134个被宣布无效），并将其归因于化验室错误。尽管一些调查并未清楚说明所发生的化验室错误，但你们并未实施全面调查来彻底审核潜在的生产原因，评估商业化历史，以识别出大量波动或OOS结果的类似情况。   For example, you opened laboratoryinvestigationOOS/C/16/IN2/FP/011 after obtaining an OOS finished product assayresult of (b)(4)%(release testing specification: (b)(4)–(b)(4)%) for(b)(4)tablets USP (b)(4) mg, batch (b)(4). You discarded the originalvial thatyielded the OOS result, which violates your OOS procedure. Testing ofstock solutions,including (b)(4) and re-dilution, yielded slightlyhigher passing results((b)(4)%, (b)(4)%, (b)(4)%). Basedon a triplicate retest, you invalidated theinitial failing result withoutinvestigating the potential manufacturing rootcauses. 例如，你们在得到XX片剂产品含量结果XX%之后（放行标准为XX-YY%）启动了化验室调查OOS/C/16/IN2/FP/011。你们抛弃了产生OOS结果的小瓶，这是违反你们的OOS程序的。对贮备液进行检测，包括XX和重新稀释液，得到结果略高于标准要求。根据3次复测结果，你们未调查潜在的生产根本原因即宣布初始不合格结果无效。   You had also obtained a low assay result for batch(b)(4),and again reported passing retest results without an investigation ofpotentialmanufacturing causes of the OOS assay result. 你们也得到XX批的低含量结果，也是未对OOS含量结果进行潜在生产原因调查即重新报告了合格的复测结果。   Your CAPA have often been limited to retrainingyouranalysts. Improvements in analytical methods and equipment were not generallyimplementedto enhance robustness and prevent errors. 你们的CAPA通常局限于重新培训化验员。一般也不对分析方法和设备进行改进来提高耐用性，防止错误。   In your response, you committed to track and trendOOSresults to identify specific tests and analysts who may be sources of therootcause. Additionally, you stated that your process for invalidating anOOSresult and accepting retest results will be more rigorous. 在你们回复中，你们承诺要追踪OOS结果，并进行趋势分析来识别出具体的检测和可能是根本原因来源的化验员。另外，你们声明说你们宣布OOS结果无效的程序和接受复测结果会更为严格谨慎。   We acknowledge your improvements in human error riskmitigationand tracking OOS trends to identify more sustainable solutions. Wealso notethat you are endeavoring to improve your OOS procedures. However,your responsewas inadequate because your investigation procedure did notappear to besufficiently remediated. You also did not perform a retrospective assessment toidentify all OOS results which were invalidated without strongscientificjustification and clear evidence. Furthermore, your response did notreviewproduction to determine whether OOS results were due to potentialproblems inmanufacturing (e.g., in-process hold times), rather than an assumedlaboratorycause. 我们了解你们改善了人为错误风险降低和追踪OOS趋势分析来识别出更为持续的解决方案。我们也注意到你们努力在改进你们的OOS程序。但是，你们的回复是不充分的，因为你们的调查程序弥补的不够。你们也没有实施回顾性评估来识别出所有的无强大科学论证和清晰证据即被宣布无效的OOS结果。另外，你们的回复未对生产进行审核以确定是否OOS是由于生产中潜在的问题（例如中间体保存时长），而不是所假定的化验室原因。 ——印度Lupin Indore   2016年6月16日， FDA对印度 Lupin 公司一所设施的检查中观察到的批检验中无法解释的高差异率提出担心。该 483 中的第一个观察项关注未解释的差异，强调 Lupin 两年内在某些批放行和稳定性测试中的超标（OOS）结果中 87% 无效。   2017年4 月 3 日 Mylan 公司的警告信中提到其在印度Nashik 的工厂在 2016 年上半年中存在72% 无效 OOS 率。   OOS调查不充分   2018.07.05 印度Baxter (Claris Injectable Ltd.)公司，稳定性研究中出现OOS结果，没有进行充分的调查，而是将结论归结未色谱柱柱效差，但色谱峰型正常，系统适用性满足要求，实验室管理人员说，保留时间，理论塔板数，拖尾因子显示均正常，没有找到根本原因，但重新复测结果符合规定，所以就作废了起始的OOS结果。   2018.07.05 印度Baxter (Claris Injectable Ltd.)，稳定性研究的杂质项目出现OOS结果，第2名实验员重新配制样品并用原先老的色谱柱及新的色谱柱重新检测，结果老的色谱柱与新的色谱柱均出现OOS结果，但调查结论却是进样小瓶污染，这个结论没有足够的数据支持，通过复测留样，作废起始的OOS结果。   2018.07.27 中国珠海联合实验室，没有对OOS结果进行足够的调查，含量项目一个批次出现OOS结果，结果偏低较多，另一批次，再次得到OOS结果，简单的调查之后，就将结论归结到玻璃器皿没有彻底清洁，并作废了OOS结果，并将2个批次的样品进行了放行。   阴阳实验室记录，隐瞒OOS结果   Dual sets of laboratory records and uninvestigatedOOS results 2套化验室记录和未经调查的OOS结果   Our investigator also found that you failed todocument, investigate, and resolve out-of-specification (OOS) results in yourlaboratory. The investigator identified two sets of laboratory testing recordsfor four (b)(4) batches and five (b)(4) batches: one set of records includedOOS results; the second set included results within specifications. You couldnot provide evidence to support the passing results. You also failed to conductinvestigations for the OOS results. Your quality department acknowledged thispractice during the inspection. 我们的调查员发现你们未记录、调查和解决你们化验室中的OOS结果。调查员发现4批次XX和5批次XX均有2套化验室检验记录：一套记录中有OOS结果，第二套记录里的结果则符合标准。你们未能提供证据支持合格结果。你们亦未对OOS结果进行调查。你们的质量部门知晓了检查中发现的这些事情。 ——Warning Letter320-18-51吉林舒兰合成药业股份有限公司   通过简单复测判定OOS结果无效   Your original atomicabsorption analysis of (b)(4)sample 15/0871 was out-of-specification(OOS). A retest of the sample was alsoOOS. A third sample was retested andfound within specifications. Youinvalidated the OOS results withoutjustification or documented investigation. 你们对XX样品15/0871的原始原子吸收分析结果不合格OOS，对该样品复测结果也是OOS。第三个样品进行了复测，发现符合质量标准。你们宣布OOS结果无效，但没有论证或书面调查。 ——Warning Letter320-18-38 法国Quali-Controle&Quali-Controle C.E.BAC   滥用假设检验/离群值检验判定OOS结果无效   Our review of your out-of-specification (OOS)investigations found that you lacked adequate procedures for investigating, andscientific justification to invalidate, OOS results. 我们审核了你们的OOS调查发现你们缺乏充分的调查程序和无效OOS的科学论证。   OOS Results for Assay 含量OOS结果   You initiated an investigation of an initial OOSassay result for (b)(4) batch (b)(4), which was found to be significantly belowspecification ((b)(4)–(b)(4)%). You also initiated an investigation of aninitial OOS assay result for (b)(4) batch (b)(4), which also yielded a testresult below specification ((b)(4)–(b)(4)%). 你们对XX批次XX产品发起了一份含量严重低于标准的OOS结果调查。你们还发起对（b）（4）批次（b）（4）产品的初始OOS结果的调查，其也产生低于标准（（b）（4） - （b）（4）％）的测试结果。   In both cases, your brief investigations found noanomalies and only stated that it was possible that the sample glassware wasnot thoroughly cleaned. Although you did not identify a laboratory error andlacked scientific justification, you invalidated the OOS results. Your firmreleased both batches based on passing retests. 在这两种情况下，你们简短的调查都没有发现任何异常现象，只是说样品玻璃器皿可能没有彻底清洗。尽管你们没有发现实验室错误，也没有科学论证，你们还是判定OOS结果无效。贵公司依据复测的合格结果放行了这两批产品。   Your acceptance of the passing results based on anassumed laboratory error was insufficient to invalidate the original failingresult and conclude the investigation. 你们基于假定的实验室错误判定初始OOS结果无效，结束调查并接受复测结果是不充分的。   Re-analysis of the actual solutions, test units,and glassware is an integral part of an investigation to determine whether alaboratory error may have occurred. This assessment, in tandem with hypothesistesting if initial re-examinations do not reveal a root cause, is instrumentalin determining whether there was a causative laboratory error. Whenever alaboratory investigation lacks conclusive evidence of laboratory error, it isessential that the investigation extends to a thorough investigation ofpotential manufacturing causes. 重新分析实际溶液，测试单元和玻璃器皿以确定是否可能发生实验室错误，是调查中不可或缺的一部分。如果初步重新检查没有揭示根本原因，这种评估与假设检验相结合，有助于确定是否存在成为原因的实验室错误。任何实验室调查缺乏与实验室错误的确凿证据，都必须将调查扩展到对潜在生产原因的彻底调查。   Your response acknowledged that there was “noscientific justification or studies performed to evaluate or prove thishypothetical root cause.” 你们的回复承认“没有执行科学论证或研究来评估或证明这个假设的根本原因。”   Since our inspection, your indicated that you haveshown that the API may degrade in the presence of residual detergent inglassware. However, your response did not include your study data. 自我们检查以来，你们表示你们已证明API可能会在玻璃器皿中残留清洗剂的情况下降解。但是，你们的回复不包括你们的研究数据。   OOS Results for Residual Solvent 残留溶剂OOS结果   You initiated investigation P201611001 for aninitial OOS result of (b)(4) parts per million (ppm) in your (b)(4)residualsolvent test (specification: not more than (b)(4) ppm) for (b)(4) API batch(b)(4). The investigation did not reveal laboratory testing anomalies. Youtested another sample preparation three times and obtained results very closeto the specification upper limit ((b)(4), and (b)(4) ppm). You invalidated theinitial failing result, stating that your statistical analysis showed asignificant difference between the original value and the retest results. Yourinvestigation lacked further assessment of the root cause of the failingresult. 你们对XX批次XX原料残留溶剂检验XXppm的初始OOS结果发起调查。调查未查出实验室异常。你们检验了另一份样品三次并得到非常接近标准上限的结果。你们判定初始OOS结果无效，并声明你们的统计学分析显示初始值与复测值有显著差异。你们的调查未能进一步评估不合格结果的根本原因。   You released the batch to use as an intermediate inyour in-house production of (b)(4) batches of (b)(4) API (batches (b)(4)). 你们放行了这个批次作为中间体用于XX批XX原料药的生产。   It is not appropriate to use an “outlier test” toinvalidate your API test results. Such statistical treatments do not identifythe cause of an extreme observation and are only of informational use. In thiscase, your investigation included multiple retests that were near the upperlimit of (b)(4) ppm, similar to the original OOS result. 使用离群值检验来判定API检验结果无效是不恰当的。这种统计学处理未能确定极端原因，仅供信息使用。在此事例中，你们的调查包括多个接近标准上限的复测结果，其实与初始OOS结果相似。   Furthermore, your OOS investigation procedure,Q0100012.001, was inadequate because it did not adequately address the need toretest the original sample and specify when a new sample should be tested. 此外，你们的OOS调查规程也不充分，因为它没能充分满足复测原始样品的要求并规定何时应该检验新的样品。   We acknowledge receipt of your revised OOSinvestigation procedure. However, your response is inadequate because it doesnot meet CGMP. Your response stated that you can use an outlier test indetermining whether to “waive the requirement for conducting appropriatelaboratory investigation to determine definitive or potential root cause(s) forthe atypical result(s).” It is inappropriate for your procedure to permitwaiver of this requirement. Your OOS procedure should specify that outliertests cannot be used for anything other than auxiliary, informational purposes. 我们收到了你们修订后的OOS调查规程。但是，你们的回复是不充分的，因为它不符合CGMP。你们的回复说你们还是允许使用离群值检验来决定是否“免去适当的实验室调查以确定异常结果的确切或潜在根本原因的要求。”你们的规程允许免去这些要求是不恰当的。你们的OOS规程应规定离群值检验不能用于辅助，信息目的之外的任何其他方面。   Your response also indicated that your firm wasretrospectively assessing effects of previously-reported OOS results on yourproducts. However, your response did not provide related records to documentyour review or summarize findings. It is unclear whether the retrospectivereview included an evaluation of your use of the statistical outlier test toinvalidate OOS results. 你们的回复还表明贵公司正在回顾性地评估先前报告的OOS结果对产品的影响。但是，你们的回复未提供相关记录以证实你们的回顾或总结结果。目前尚不清楚该回顾性审查是否包括对使用统计学离群值检验判定OOS结果无效的评估。   OOS根本原因与相关记录自相矛盾   Our review of yourout-of-specification (OOS)investigations found that you did not use adequateOOS procedures, and lackedscientific justification to invalidate initial OOSresults. For example: 我们对你们的OOS调查审核发现你们并未使用充分的OOS程序，缺乏科学论断来判定初始OOS结果无效。例如：   a. Investigation reportOOS/2015/098 was initiated for an initial OOS resultin your related substancestest, where (b)(4)% and (b)(4)% (specification: notmore than (b)(4)%)was obtained for Impurity (b)(4) in (b)(4)API batches(b)(4)and (b)(4), respectively. Your investigation concludedthatover-sonication might have increased the temperature of the water bathandcaused degradation of the sample solution. 调查报告OOS/2015/098由于一个有关物质检测的初始OOS结果启动，某API批次检测的杂质结果分别为XX（质量标准为不得过YY）和ZZ。你们的调查结论是超声过度可能会增加水浴温度，导致样品溶解降解。   However, your investigationlacked evidence tosupport this possible root cause. Instead, yourinvestigation found that theanalyst only briefly sonicated the solution (forabout (b)(4)) at (b)(4)temperature. In addition, degradationstudies conducted as part of highperformance liquid chromatography (HPLC)validation showed that heat degradationwas minimal even after (b)(4) atextremely high ((b)(4)o C) temperatures. 但是，你们的调查缺乏证据来支持该可能的根本原因。相反，你们的调查发现化验员对溶液的超声时间很短（大约XX时间），温度为XX。另外，作为HPLC验证的一部分所实施的降解研究显示即使在极高温度XX下其热降解仍非常之小。   Although your investigationwas inconclusive, youdid not proceed to Phase 2 and investigate potentialcauses of the OOS resultrelating to deficient manufacturing and productquality. 尽管你们的调查并无法得出结论，你们并未进入第二阶段，调查与生产缺乏和产品质量有关的潜在OOS结果原因。 ——Warning Letter320-18-12 德国Fresenius KabiAG   OOS原因未进行证实，仅通过复测代替原结果   Investigation reportOOS 50989 was initiatedfollowing initial OOS results for “relatedsubstances–unspecified impurities”for (b)(4) API stability batches (b)(4)and (b)(4). You concluded that the mostprobable cause of the OOSresult was contamination, although the source of thecontamination was notidentified or confirmed through your hypothesis study. Youinvalidated the OOSresults (as well as an additional failing retest from afresh sample preparationby a second analyst for batch (b)(4)) and reported the averageof sixretests. You failed to expand the investigation to review potentialcauses ofthe OOS result relating to deficient manufacturing and productquality. 调查报告OOS50989启动是因为XX原料药稳定性批准XX和YY的“有关物质—未知杂质”初始结果为OOS。你们得出结论说该OOS结果最可能的原因是污染，虽然污染来源并未识别出来，也未通过你们的假设性研究得到证实。你们宣布该OOS结果无效（由第二个化验员对XX批制备新鲜样品进行复测，结果仍不合格），报告了6次复测结果的平均值。你们未能扩大调查审核与生产缺乏和产品质量有关的潜在OOS结果原因。 ——WarningLetter 320-18-12 德国Fresenius KabiAG   OOS调查规程不正确，投机取巧   Your OOSinvestigation procedure 036/—/QS/QA permitsan analyst to abort achromatographic run if an apparent OOS is observed priorto completing analysisof all samples scheduled to be injected in the sequence.Your quality control(QC) manager confirmed that analysts abort HPLC analyses ifthey “expect toinvalidate” them later for an assignable cause. For example, youaborted theHPLC sequence of (b)(4) API batch (b)(4) while observingthechromatographic run on the screen (“online monitoring”) in which anindividualunknown impurity tested at (b)(4)% (specification: NMT (b)(4)%).Therewas no machine malfunction (e.g., unstable system) that would justifyabortingthe automated analysis. 你们的OOS调查程序0366/—/QS/QA允许化验员在发现明显OOS结果但尚未完成该序列中所有样品分析之前中断色谱运行。你们的QC经理确认化验员如果他们“期望之后由于可归结的原因宣布无效”则中断HPLC分析。例如，你们在屏幕上（在线监测）发现色谱运行中出现一个单一未知杂质达到XX%（标准为不得过YY%）时，你们中断了XX原料药批次的HPLC序列。没有设备故障（例如，系统不稳定）支持中断自动分析过程。   Our investigators documentedapproximately 248instances of aborted sequences. 我们调查人员记录下了约248次中断的序列。   Your SOP was inadequate. Whenperforming a samplepreparation, it may be possible to identify an obviousmanual error at the timeof the mistake. In such a limited instance, it can beappropriate to discontinuethe sample preparation, immediately document thedeviation, and justify a newsample preparation. However, it is not appropriateto stop an in-progressautomated analysis because of an assumption that anearlier error may be causingan OOS result. Obtaining an unexpected result doesnot constitute an “assignablecause” and the assumption of such a cause is nota valid basis for interruptingan analysis. The automated analytical sequenceshould be allowed to proceed tocompletion, irrespective of the appearance ofundesirable analytical results onthe computer screen. 你们的SOP是不充分的。在执行一个样品制备时，发生错误时可能会发现明显的人为错误。在此类有限情形下，中止样品制备是恰当的，立即记录下偏差，论证新的样品制备。但是，由于假定更早的一个错误可能导致一个OOS结果从而停止一个正在进行的自动化分析则是不恰当的。得到一个非预期的结果并不构成一个“可归结的原因”，此类原因的假设并不是中断分析的有效基础。应允许自动化分析序列继续直到完成，而不管计算机屏幕上出现的是否所想要的分析结果。   We acknowledge your commitmentto correct thisdeficient SOP. Your response was inadequate because yourcorrections did notensure that lab investigations will be started immediatelyafter obtaining anOOS result. You acknowledged that in about nine of theexamples referenced bythe investigator, the original samples were notre-injected due to samplesolution stability. Notably, your method validationdata show that some of thesesample solutions are stable for up to (b)(4) atroom temperature. Promptre-testing of the actual stock, working, and HPLC vialsolutions is essential todetermine if mechanical error or preparation errormay have occurred. Timelyinvestigations of potential original laboratorysample preparations areessential to provide clear evidence and credibility forlaboratory errorhypotheses. 我们了解你们承诺要纠正该有缺陷的SOP。你们的回复是不充分的，因为你们的纠正措施并不能确保化验室调查会在得到OOS结果之后立即开始。你们说在调查员引用的大概9个例子中，由于样品溶液稳定性原因，原始样品并未重新进样。显然，你们的方法验证数据显示有些样品溶液在室温下稳定长达XX时间。立即重新检测实际贮备溶液、工作液和HPLC小瓶中的溶液对确定是否有可能发生机械性错误或制备错误是很基本的。对可能的原始化验室样品制备进行及时调查对于提供清楚证据和化验室错误假设可信度是很重要的。   Your response was alsoinadequate because your OOSprocedure failed to ensure that you proceed toPhase 2 whenever you lackconclusive evidence of laboratory error. A possiblelaboratory error isinsufficient to close an investigation at Phase 1. Inaddition, your procedureindicated that you can close an investigation when asecond analyst confirms theinitial OOS without moving to a Phase 2investigation. It remains unclearwhether all failing results would beinvestigated for their manufacturing rootcauses prior to closing aninvestigation. Further, even if an OOS result is notconfirmed by a secondanalyst, it should not be assumed that the initial OOStest result wasattributable to analytical error. Whenever an investigationlacks conclusiveevidence of laboratory error, a thorough investigation ofpotentialmanufacturing causes must be performed. 你们的回复不充分还因为你们的OOS程序未能确保你们在缺乏可得出化验室错误结论的证据时进入第二阶段。可能的化验室错误对于在第一阶段就关闭调查是不充分的。此外，你们的程序说如果第二个化验员确认初始OOS，则你们可以关闭该调查而不需要进入第二阶段调查。这里没写清楚是否所有不合格结果都需要调查其生产根本原因，方可关闭调查。还有，即使一个OOS结果没有被第二个化验员证实，也不应该假定初始的OOS结果应归因于分析错误。如果一个调查缺乏能给出结论的证据证明化验室错误，则必须对潜在的生产原因进行彻底调查。 ——Warning Letter320-18-12 德国Fresenius KabiAG   尽管OOS相关批次被拒签，但未进行实验室和生产调查   You obtained an initialout-of-specification (OOS)viscosity test result for (b)(4) Cream lot (b)(4).The next two retest valueswere also OOS. You failed to conduct laboratory andmanufacturing investigationsinto these OOS results, which included identifyinga root cause and implementingcorrective actions and preventive actions (CAPA).Instead, you rejected thebatch without conducting an adequate investigation. 你们在XX霜批次XX的粘度检测中得到初始的OOS结果。接下来2次复测值也是OOS。你们未对这些OOS结果展开实验室和生产调查，包括识别出根本原因和实施纠正措施和预防措施（CAPA）。相反，你们没有进行充分的调查即拒收了该批次。 ——Warning Letter320-18-16 韩国AN Co. Ltd   将个别OOS识别为事故而未进行调查   We also found that youinvestigated numerous OOSresults between February 2015 and April 2017 as“incidents” and not as OOSresults. Your “incident” procedure did not require asubstantive investigationof OOS results. Your response acknowledgesthat this procedure was inadequateand that consequently your decisionsregarding OOS results were not supported bysufficient inquiry and scientificrationale.   我们也发现你们在2015年2月至2017年4月间调查了大量的OOS结果，将其作为“事故”而不是作为OOS结果。你们的“事故”程序并不要求对OOS结果进行实质性调查。你们的回复告知说该程序是不充分的，因此你们对于OOS结果的判定没有充分的询问和科学合理性来支持。 ——Warning Letter320-18-41印度KeshavaOrganics   During the inspection, our investigator reviewedthe electronic HPLC injection history for (b)(4) intermediate stability sample,batch (b)(4). The history indicated that the same vial was injected twice onJune 14, 2017. The first injection was not included in the final data packetprovided to the quality unit for batch review, and the intermediate batch wasultimately cleared for and used in manufacturing a finished lot of (b)(4) API,batch (b)(4). 在此次检查中我们检查人员审核了XX中间体XX批号稳定性样品的电子HPLC进样历史。该历史显示相同样品瓶在2017年6月14日进样2次。第一次进样未包括在提交给质量部门用于批审核的最终数据包里，中间体批最终放行并用于生产批次XX的原料药，批号为XX。   According to your quality control analyst, thefirst injection appeared abnormal because it did not show a peak at theexpectedretention time. The second injection, within specification, was used toreleasethe batch. There was no documentation, explanation, or investigation oftheabnormal result of the first injection. 根据你们QC化验员的说法，第一次进样显示不正常，因为在预期的保留时间内未出峰。第二次进样在质量标准范围内，所以用于放行该批次。第一次进样的异常结果并无文件记录、解释或调查。   Our investigator also observed similar instances inwhich abnormal injections were disregarded without investigation. 我们的检查人员还发现异常进样被忽略且未调查的类似情况。   In your response, you stated that you conducted adeviation investigation of batch (b)(4) on October 21, 2017, and you startedretesting retention samples of related batches at that time. Your response isinadequate because it lacks details of this deviation investigation.  It also lacks a comprehensive assessment andremediation of your overall system for investigations of deviations, atypicalevents,complaints, out-of-specification results, and failures. 在你们的回复中，你们声称你们在2017年10月21日对XX批次进行了偏差调查，你们对当时的相关批次开始进行留样复测。你们的回复是不充分的，因为它缺乏该偏差调查的详细内容，还缺乏对你们偏差调查、异常事件、投诉、OOS结果和失败全面系统的综合评估和补救措施。 ——丽江映华生物药业有限公司   未知杂质峰，归因为实验室错误   Invalidation of out-of-specificationresults lacksadequate scientific justification.  宣布OOS结果无效时缺乏充分的科学论证。 a)    ReportOOS-CQC15067 relating to (b)(4)batch (b)(4) was reported “Unknown impurity peakis appeared under unknownreason”. Your firm explained this unknown peak as a“ghost peak” that appearsfrom time to time in chromatograms for undeterminedreasons. Without anindication of the cause of the out-of-specification, anattribution of “Laberror wasmade.”  报告OOS-CQC15067是XX产品批次XX，报告“未知杂质峰出现，原因不明”。你们公司将此未知峰解释为“鬼峰”，因未知原因偶尔出现在色谱图中。未指明OOS的原因，归因于“实验室犯错”。 b)   ReportOOS-CQA16103 reportedout-of-specification of residual solvents in (b)(4). ThePhase I laboratoryinvestigation failed to identify a laboratory error. Thisinvestigationattributed the failure to “Pollution” from the environment duringsamplepreparation. 报告OOS-CQA16103是XX中残留溶剂OOS结果。第一阶段实验室调查未能发现实验室错误。该调查将不合格归因于样品制备过程中来自环境的“污染”。 c)    ReportOOS-CQC15103 due to a singleimpurity in (b)(4) batch (b)(4) ((b)(4)% against aspecification of no morethan (b)(4)%). This was assigned as a “Lab error” dueto “possible” residue inthe column. When inquiring about why this impurityspecifically eluted in the(b)(4) analytical test of the testing sequence, yourfirm again referenced a“ghost peak”.  报告OOS-CQC15103是XX产品XX批次单杂（结果为XX%，标准为不得过XX%）。该OOS归因为因柱中“可能”有残留而发生的“实验室错误”。在询问为何在检测序列中的XX分析测试中会有洗出该特定杂质时，你们公司又将其归为“鬼峰”。 ——浙江华海药业   OOS调查情况与回答问题不一样   （E）     Your firm has invalidatedseveral Out-of-Specification (OOS) results obtained during the testing ofenvironmental monitoring as summarized below: You attributed several of them tohuman error which raises concerns about the ability of your laboratorypersonnel to properly conduct the required analytical testing. For example: 你公司宣布在环境监测期间得到的几个OOS结果无效，总结如下。你们将其中几个归结为人为错误，这使得我们担心你们实验室人员正确执行所需分析检测的能力。例如： •        OOS investigation, EMO/OOS/18-001 wasinitiated on 04/15/2018 for routine air monitoring sampling activity (nofilling) where the result for Location #(b)(4) found exceeding the acceptancelimits (b)(4) CFU (Action/Level Limit (b)(4) CFU). Your firm attributed theroot cause as human error through a deficient investigation and interview wherethe sampler lid might have not been properly sanitized before sampling in(b)(4) ((b)(4)). When interviewing the Analyst (Officer Micro) duringtheinspection on 01/21/2019 using the same questions raised in theinvestigation report, his response did not corroborate what is stated in theOOS investigation report conducted by your firm. For example, in the OOS reporthe said for Question 6 that “I have taken plates from PPM staging and loaded at(b)(4) trolley. After loading, air sampling was performed as per sequencementioned below: (b)(4). Again entered in Aseptic area and performed settleplate exposer following the sequence as above in each step, I have sanitizedthe hand while (b)(4)”, but in person on 01/21/2019 he said “Firstly, sequenceof (b)(4), it mentions in our SOP (QC2-128-13).” Your firm invalidated thefailing result through your interview process. •        OOS调查EMO/OOS/18-001于2018年4月15日启动，在XX位置常规空气监测取样活动中（未灌装）发现超出可接受标准XXCFU（行动限XXCFU）。你们的调查和面谈都是有缺陷的，你们的取样品盖可能在XX取样之前未进行恰当消毒，但你公司将根本原因归结为人为错误。在20190121检查期间与化验员（微生物管理员）面谈时，我们提出了与调查期间所提的相同问题，化验员的回答与你公司所做OOS调查报告中记载的不同。例如，在OOS报告中，他对问题6的回答是“我从PPM阶段取了碟子，放到XX推车上。放好后，按以下顺序执行取样。再次进入无菌区，按上述各步骤打开沉降碟采样，我在XX时进行了手消毒”。但在20190121，他说的是“首先，按顺序XX，在我们SOP（QC2-128-13）中有提及”。你们公司通过你们面谈过程将不合格结果宣布无效。 ——Lupin Limited   OOS原因经不起推敲 Investigation OOS 310013508 opened for (b)(4)OOS results above the release specification of not more than (b)(4) ppm for lots (b)(4) of (b)(4) tablets was not thorough and did not follow procedure SOP GQA035 “Handling Out of Specification Results”. The investigation identified the use of (b)(4) tubes as a probable root causes, but could not confirm (b)(4) tubes were used in the preparation of the OOS samples. The investigation did not evaluate why the sample from lot (b)(4) that was prepared at the same time as the OOS samples did not detect any impurity. OOS调查310013508是针对某批次片剂中XX超过放行标准，其调查不彻底，未遵守SOP GQA035“OOS结果处理”。调查认为使用XX管是可能的根本原因，但并不能确认在OOS样品制备中是否使用了XX管。调查并未评估为何与OOS样品同时制备的其它样品中未发现任何杂质。 ——Dr. Reddy’s Laboratories Limited