The portuguese public procurement system as a strategic instrument of public management
DOI:
https://doi.org/10.22267/rtend.222302.221Keywords:
administrative sciences, factor analysis, Portugal, public administration, public sectorAbstract
This research aims to evaluate the National Public Procurement System (NPPS) as a strategic instrument of public management, in Portugal. The creation of NPPS, in 2007, followed the European Union model for public procurement, based on criteria of economy, efficiency, and effectiveness that leads to greater transparency and accountability and reflects public value. This experiment analysis which are the weakest and the strongest dimensions of the NPPS and to what extent does the performance of the NPPS fit as a strategic instrument of public management. A research survey, based on the conceptual model developed by Essig et al. (2010) and later applied by Thi et al. (2012), adapted to Portuguese context, was applied to 1,500 entities linked to the NPPS. The sample size of 101 valid questionnaires corresponds to an effective response rate of 5.6%. The data analysis used exploratory factor analysis to validate the distribution of the variables and performed a principal component analysis with varimax rotation. The research concluded that NPPS has, in general, a low level of performance, despite the positive contribution of the “Information and control” and “Electronic platforms” factors in process management. These results corroborate the need for a realignment of the public procurement strategy in Portugal.
Downloads
Download data is not yet available.
References
(1) Albano, G. L. & Sparro, M. (2010). Flexible Strategies for Centralized Public Procurement. Review of Economics and Institutions, 1(2), 1-32.
(2) Albano, G. L., Snider, K. F. & Thai, K. V. (2013). Charting a Course in Public Procurement Innovation and Knowledge Sharing. In G. L. Albano, K. F. Snider. & K. V. Thai (Eds.), Charting a Course in Public Procurement Innovation and Knowledge Sharing (pp. 1-27). PrAcademics Press.
(3) Andrecka, M. (2015). Framework agreements, EU procurement law and the practice. Procurement Law Journal, (2), 127-149.
(4) Arrowsmith, S. (2009). EC Regime on Public Procurement. In K. V. Thai (Ed.), International handbook of public procurement (pp. 252-285). CRC Press.
(5) Arrowsmith, S. (2014). The Law of Public and Utilities Procurement. Regulations in the EU and UK (3rd ed.). Sweet & Maxwell.
(6) Bovaird, T. (2016). Strategic management in public sector organizations. In T. Bovaird & E. Loeffler (Eds.), Public Management and Governance (pp. 57-75). Routledge.
(7) Boyne, G. A. & Walker, R. M. (2010). Strategic management and public service performance: the way ahead. Public Administration Review, 70, S185-S192.
(8) Bozeman, B. (2007). Public values and public interest: Counterbalancing economic individualism. Georgetown University Press.
(9) Callender, G. & Matthews, D. (2002). The economic context of government procurement: new challenges and new opportunities. Journal of Public Procurement, 2(2), 216-236.
(10) Caupers, J. (2013). Contratação Pública e Concorrência. In C. Trabuco & V. Eiró (Eds.). Contratação Pública e Concorrência (pp. 7-19). Almedina.
(11) Court of Auditors. (2015). Follow-up audit of the National Public Procurement System (motor vehicles and motorcycles and hygiene and cleaning). Audit Report No. 25/2015 - 2nd Section. Court of Auditors.
(12) Court of Auditors. (2020). Audit of Ecological Public Purchases. Audit Report No. 7/2020 - 2nd Section. Court of Auditors.
(13) Decree-Law 18. (2018, 29 of January). Amended and republished by Decree-Law No 111- B/2017, of 31 August (rectified by the declarations of rectification No. 36-A/2017, of October 30 and No. 42/2017, of November 30), Public Contracts Code, Published in the Portuguese Official Gazette.
(14) Decree-Law 37. (2007, 19 of February). Creates the National Public Procurement Agency and approves the respective statutes, Published in the Portuguese Official Gazette No. 3.
(15) Demircioglu, M. A. & Vivona, R. (2021). Positioning public procurement as a procedural tool for innovation: an empirical study. Policy and Society, 40(3), 379-396.
(16) Dunleavy, P. & Hood, C. (1994). From Old Public Administration to New Public Management. Public Money & Management, 14(3), 9-16.
(17) Edler, J. & Georghiou, L. (2007). Public Procurement and Innovation-Resurrecting the Demand Side. Research Policy, 36(7), 949-963.
(18) Erridge, A. (2007). Public Procurement, Public Value and the Northern Ireland Unemployment Pilot Project. Public Administration, 85, 1023-1043.
(19) Erridge, A. & Greve, C. (2016). Contracting for public services. In T. Bovaird & E. Loeffler (Eds.), Public management and governance (pp. 90-103). Routledge.
(20) Essig, M., Amann, M. & Boerner, R. (2010, August 26-28). Conceptual Framework for Performance Measurement in Public Procurement [Paper presentation]. Fourth International Public Procurement Conference, Seoul, South Korea.
http://www.ippa.org/images/PROCEEDINGS/IPPC4/12ProcurementPerformance/Paper12-3.pdf
(21) Essig, M., Dorobek, S., Glas, A. & Leuger, S. (2009). Public Procurement in Germany. In K.V. Thai (Ed.), International Handbook of Public Procurement (pp. 307-322). Taylor & Francis Group.
(22) European Commission. (2011). Green Paper on the modernisation of EU public procurement policy Towards a more efficient European Procurement Market. European Commission.
(23) European Commission. (2018). Public Procurement Guidance for Practitioners on avoiding the most common errors in projects funded by the European Structural and Investment Funds. EU Publications.
(24) Government of Portugal. (2015). Program of the XXI Constitutional Government 2015-2019.
(25) Government of Portugal. (2019). Program of the XXII Constitutional Government 2019-2023.
(26) Grandia, J. (2018). Public Procurement in Europe. In E. Ongaro & S. V. Thiel (Eds.), The Palgrave Handbook of Public Administration and Management in Europe (pp. 363-379). Palgrave Macmillan.
(27) Guarnieri, P. & Gomes, R. (2019). Can public procurement be strategic? A future agenda proposition. Journal of Public Procurement, 19(4), 295-321.
(28) Halonen, K. (2019). Many faces of transparency in public procurement. In K-M. Halonen, R. Caranta & A. Sanchez (Eds.), Transparency in EU Procurements. Disclosure Within Public Procurement and During Contract Execution (pp. 8-32). Edward Elgar Publishing.
(29) Harland, C., Telgen, J., Callender, G., Grimm, R. & Patrucco, A. (2019). Implementing Government Policy in Supply Chains: An International Coproduction Study of Public Procurement. Journal of Supply Chain Management, 55(2), 6-25.
(30) Hill, M. M. & Hill, A. (2012). Investigação por questionário (2nd ed. rev.). Sílabo Publishing.
(31) Hood, C. (1991). A Public Management for All Seasons? Public Administration Review, 69, 3-19.
(32) Hood, C. (1995). The ‘New Public Management’ in the 1980’s: variations on a theme. Accounting, Organizations and Society, 20(2/3), 93-109.
(33) Hudon, P. A., Dumas, P. & Garzón, C. (2021). Recent research on public procurement: Should it become a subfield of public administration? Canadian Public Administration, 64(2), 179-201.
(34) Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC). (2021). Contratação Pública em Portugal. Relatório anual 2020.
(35) Karjalainen, K. (2011). Estimating the cost effects of purchasing centralization - Empirical evidence from framework agreements in the public sector. Journal of Purchasing and Supply Management, 17(2), 87-97.
(36) Kelman, S. (2002). Remaking federal procurement. Public Contract Law Journal, 31(4), 581-622.
(37) Knutsson, H. & Thomasson, A. (2014). Innovation in the Public Procurement Process: A study of the Creation of Innovation Friendly Public Procurement. Public Management Review, 16(2), 242-255.
(38) Kuhlmann, S. & Wollmann, H. (2014). Introduction to Comparative Public Administration: Administrative Systems and Reforms in Europe (1st ed.). Edward Elgar.
(39) Lopes, C. S. A. & Romão, A. L. (2021). Public Procurement as an Instrument for Transparency and Competition of Public Procurement in Portugal. In D. Ribeiro, A. I. Rodrigues, I. Pinho & P. López (Eds.), New Trends in Qualitative Research (pp. 267-277). Ludomedia Publishing.
(40) Matthews, D. (2005). Strategic procurement in the public sector: A mask for financial and administrative policy. Journal of Public Procurement, 5(3), 388-399.
(41) McCue, C., Prier, E. & Swanson, D. (2015). Five Dilemmas in Public Procurement. Journal of Public Procurement, 15(2), 177-207.
(42) Moore, M. H. (1995). Creating public value: Strategic management in government. Harvard University Press.
(43) Moore, M. H. (2000). Managing for Value: Organizational Strategy in For-Profit, Nonprofit, and Governmental Organizations. Nonprofit and Voluntary Sector Quarterly, 29(1), 183-204.
(44) O’Flynn, J. (2007). From New Public Management to Public Value: Paradigmatic Change and Managerial Implications. The Australian Journal of Public Administration, 66(3), 353-366.
(45) Organisation for Economic Co-operation and Development. (2015). OECD Recommendation of the Council on Public Procurement. OECD Publishing.
(46) Organisation for Economic Co-operation and Development. (2016). Fighting bid rigging in public procurement: Report on implementing the OECD Recommendation. OEDC Publishing.
(47) Organisation for Economic Co-operation and Development. (2019). Public Procurement in Germany: Strategic Dimensions for Well-being and Growth. OECD Publishing.
(48) Organisation for Economic Co-operation and Development. (2021). Government at a Glance 2021. OECD Publishing.
(49) Plantinga, H., Voordijk, H. & Dorée, A. (2020). Clarifying strategic alignment in the public procurement process. International Journal of Public Sector Management, 33(6/7), 791-807.
(50) Pollitt, C. & Bouckaert, G. (2011). Public Management Reform: A Comparative Analysis - New Public Management, Governance, and the Neo-Weberian State (3rd ed.). Oxford University Press.
(51) Pollitt, C., Thiel, S. & Homburg, V. (2007). New Public Management in Europe. Management Online Review, 1-7.
(52) Sánchez, A. (2015). Public Procurement and the EU Competition Rules. (2nd ed.) Hart Publishing.
(53) Schapper, P., Veiga Malta, J. & Gilbert, D. (2006). An Analytical Framework for the Management and Reform of Public Procurement. Journal of Public Procurement, 6(1), 1-26.
(54) Secchi, L. (2009). Modelos organizacionais e reformas da administração pública. Revista de Administração Pública, 43(2), 347-369.
(55) Snider, K. & Rendon, R. (2008). Public procurement policy: Implications for theory and practice. Journal of Public Procurement, 8(3), 310-333.
(56) Snider, K. & Rendon, R. (2012). Public Procurement: Public Administration and Public Service Perspectives. Journal of Public Affairs Education, 18(2), 327-348.
(57) Tátrai, T. (2015). Debate: Cutback Management Through Public Procurement. Public Money & Management, 35(4), 255-256.
(58) Thai, K. V. (2001). Public Procurement Re-Examined. Journal of Public Procurement, 1, 9-50.
(59) Thai, K. V. (2009). International Public Procurement: Concepts and Practices. In International Handbook of Public Procurement (pp. 1-22). Taylor & Francis Group.
(60) Thai, K. V. (2017). Global Public Procurement Theories and Practices: An Introduction. In Global Public Procurement Theories and Practices (pp. 1-14). Springer International Publishing.
(61) Thai, K.V., Araujo, A., Carter, R. Y., Callender, G. & Telgen, J. (2005). Challenges in Public Procurement:an international perspective (1st ed.). PrAcademics Press.
(62) The European Parliament and the Council of the European Union. (2014). Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on Public Procurement and Repealing Directive 2004/18/EC. Official Journal of the European Union, (L 94), 65-242.
(63) The Government Shared Services Entity (eSPap). (2022). Public Procurement Shared Services. Retrieved April 10, 2022.
www.espap.gov.pt/en/spcp/Pages/spcp.aspx#maintab3
(64) Thi, T., Essig, M. & Amann, M. (2012, August 17-19). Public Procurement Benchmarking: Conceptual Model and First Empirical Findings [Paper presentation]. Fifth International Public Procurement Conference, Seattle, USA.
http://www.ippa.org/images/PROCEEDINGS/IPPC5/Part5/PAPER5-7.pdf
(65) Trepte, P. (2005). Transparency and accountability as tools for promoting integrity and preventing corruption in procurement: Possibilities and limitations. OECD Papers, 5(10), 1-37.
(66) Vaidya, K., Sajeev, A. & Callender, G. (2006). Critical factors that influence e-procurement implementation success in the public sector. Journal of Public Procurement, 6(1-3), 70-99.
(67) Walker, H. & Brammer, S. (2009). Sustainable procurement in the United Kingdom public sector. Supply Chain Management: An International Journal, 14(2), 128-137.
(68) World Bank Group. (2016). Benchmarking Public Procurement 2017. Assessing public procurement regulatory systems in 180 economies. The World Bank.
(2) Albano, G. L., Snider, K. F. & Thai, K. V. (2013). Charting a Course in Public Procurement Innovation and Knowledge Sharing. In G. L. Albano, K. F. Snider. & K. V. Thai (Eds.), Charting a Course in Public Procurement Innovation and Knowledge Sharing (pp. 1-27). PrAcademics Press.
(3) Andrecka, M. (2015). Framework agreements, EU procurement law and the practice. Procurement Law Journal, (2), 127-149.
(4) Arrowsmith, S. (2009). EC Regime on Public Procurement. In K. V. Thai (Ed.), International handbook of public procurement (pp. 252-285). CRC Press.
(5) Arrowsmith, S. (2014). The Law of Public and Utilities Procurement. Regulations in the EU and UK (3rd ed.). Sweet & Maxwell.
(6) Bovaird, T. (2016). Strategic management in public sector organizations. In T. Bovaird & E. Loeffler (Eds.), Public Management and Governance (pp. 57-75). Routledge.
(7) Boyne, G. A. & Walker, R. M. (2010). Strategic management and public service performance: the way ahead. Public Administration Review, 70, S185-S192.
(8) Bozeman, B. (2007). Public values and public interest: Counterbalancing economic individualism. Georgetown University Press.
(9) Callender, G. & Matthews, D. (2002). The economic context of government procurement: new challenges and new opportunities. Journal of Public Procurement, 2(2), 216-236.
(10) Caupers, J. (2013). Contratação Pública e Concorrência. In C. Trabuco & V. Eiró (Eds.). Contratação Pública e Concorrência (pp. 7-19). Almedina.
(11) Court of Auditors. (2015). Follow-up audit of the National Public Procurement System (motor vehicles and motorcycles and hygiene and cleaning). Audit Report No. 25/2015 - 2nd Section. Court of Auditors.
(12) Court of Auditors. (2020). Audit of Ecological Public Purchases. Audit Report No. 7/2020 - 2nd Section. Court of Auditors.
(13) Decree-Law 18. (2018, 29 of January). Amended and republished by Decree-Law No 111- B/2017, of 31 August (rectified by the declarations of rectification No. 36-A/2017, of October 30 and No. 42/2017, of November 30), Public Contracts Code, Published in the Portuguese Official Gazette.
(14) Decree-Law 37. (2007, 19 of February). Creates the National Public Procurement Agency and approves the respective statutes, Published in the Portuguese Official Gazette No. 3.
(15) Demircioglu, M. A. & Vivona, R. (2021). Positioning public procurement as a procedural tool for innovation: an empirical study. Policy and Society, 40(3), 379-396.
(16) Dunleavy, P. & Hood, C. (1994). From Old Public Administration to New Public Management. Public Money & Management, 14(3), 9-16.
(17) Edler, J. & Georghiou, L. (2007). Public Procurement and Innovation-Resurrecting the Demand Side. Research Policy, 36(7), 949-963.
(18) Erridge, A. (2007). Public Procurement, Public Value and the Northern Ireland Unemployment Pilot Project. Public Administration, 85, 1023-1043.
(19) Erridge, A. & Greve, C. (2016). Contracting for public services. In T. Bovaird & E. Loeffler (Eds.), Public management and governance (pp. 90-103). Routledge.
(20) Essig, M., Amann, M. & Boerner, R. (2010, August 26-28). Conceptual Framework for Performance Measurement in Public Procurement [Paper presentation]. Fourth International Public Procurement Conference, Seoul, South Korea.
http://www.ippa.org/images/PROCEEDINGS/IPPC4/12ProcurementPerformance/Paper12-3.pdf
(21) Essig, M., Dorobek, S., Glas, A. & Leuger, S. (2009). Public Procurement in Germany. In K.V. Thai (Ed.), International Handbook of Public Procurement (pp. 307-322). Taylor & Francis Group.
(22) European Commission. (2011). Green Paper on the modernisation of EU public procurement policy Towards a more efficient European Procurement Market. European Commission.
(23) European Commission. (2018). Public Procurement Guidance for Practitioners on avoiding the most common errors in projects funded by the European Structural and Investment Funds. EU Publications.
(24) Government of Portugal. (2015). Program of the XXI Constitutional Government 2015-2019.
(25) Government of Portugal. (2019). Program of the XXII Constitutional Government 2019-2023.
(26) Grandia, J. (2018). Public Procurement in Europe. In E. Ongaro & S. V. Thiel (Eds.), The Palgrave Handbook of Public Administration and Management in Europe (pp. 363-379). Palgrave Macmillan.
(27) Guarnieri, P. & Gomes, R. (2019). Can public procurement be strategic? A future agenda proposition. Journal of Public Procurement, 19(4), 295-321.
(28) Halonen, K. (2019). Many faces of transparency in public procurement. In K-M. Halonen, R. Caranta & A. Sanchez (Eds.), Transparency in EU Procurements. Disclosure Within Public Procurement and During Contract Execution (pp. 8-32). Edward Elgar Publishing.
(29) Harland, C., Telgen, J., Callender, G., Grimm, R. & Patrucco, A. (2019). Implementing Government Policy in Supply Chains: An International Coproduction Study of Public Procurement. Journal of Supply Chain Management, 55(2), 6-25.
(30) Hill, M. M. & Hill, A. (2012). Investigação por questionário (2nd ed. rev.). Sílabo Publishing.
(31) Hood, C. (1991). A Public Management for All Seasons? Public Administration Review, 69, 3-19.
(32) Hood, C. (1995). The ‘New Public Management’ in the 1980’s: variations on a theme. Accounting, Organizations and Society, 20(2/3), 93-109.
(33) Hudon, P. A., Dumas, P. & Garzón, C. (2021). Recent research on public procurement: Should it become a subfield of public administration? Canadian Public Administration, 64(2), 179-201.
(34) Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC). (2021). Contratação Pública em Portugal. Relatório anual 2020.
(35) Karjalainen, K. (2011). Estimating the cost effects of purchasing centralization - Empirical evidence from framework agreements in the public sector. Journal of Purchasing and Supply Management, 17(2), 87-97.
(36) Kelman, S. (2002). Remaking federal procurement. Public Contract Law Journal, 31(4), 581-622.
(37) Knutsson, H. & Thomasson, A. (2014). Innovation in the Public Procurement Process: A study of the Creation of Innovation Friendly Public Procurement. Public Management Review, 16(2), 242-255.
(38) Kuhlmann, S. & Wollmann, H. (2014). Introduction to Comparative Public Administration: Administrative Systems and Reforms in Europe (1st ed.). Edward Elgar.
(39) Lopes, C. S. A. & Romão, A. L. (2021). Public Procurement as an Instrument for Transparency and Competition of Public Procurement in Portugal. In D. Ribeiro, A. I. Rodrigues, I. Pinho & P. López (Eds.), New Trends in Qualitative Research (pp. 267-277). Ludomedia Publishing.
(40) Matthews, D. (2005). Strategic procurement in the public sector: A mask for financial and administrative policy. Journal of Public Procurement, 5(3), 388-399.
(41) McCue, C., Prier, E. & Swanson, D. (2015). Five Dilemmas in Public Procurement. Journal of Public Procurement, 15(2), 177-207.
(42) Moore, M. H. (1995). Creating public value: Strategic management in government. Harvard University Press.
(43) Moore, M. H. (2000). Managing for Value: Organizational Strategy in For-Profit, Nonprofit, and Governmental Organizations. Nonprofit and Voluntary Sector Quarterly, 29(1), 183-204.
(44) O’Flynn, J. (2007). From New Public Management to Public Value: Paradigmatic Change and Managerial Implications. The Australian Journal of Public Administration, 66(3), 353-366.
(45) Organisation for Economic Co-operation and Development. (2015). OECD Recommendation of the Council on Public Procurement. OECD Publishing.
(46) Organisation for Economic Co-operation and Development. (2016). Fighting bid rigging in public procurement: Report on implementing the OECD Recommendation. OEDC Publishing.
(47) Organisation for Economic Co-operation and Development. (2019). Public Procurement in Germany: Strategic Dimensions for Well-being and Growth. OECD Publishing.
(48) Organisation for Economic Co-operation and Development. (2021). Government at a Glance 2021. OECD Publishing.
(49) Plantinga, H., Voordijk, H. & Dorée, A. (2020). Clarifying strategic alignment in the public procurement process. International Journal of Public Sector Management, 33(6/7), 791-807.
(50) Pollitt, C. & Bouckaert, G. (2011). Public Management Reform: A Comparative Analysis - New Public Management, Governance, and the Neo-Weberian State (3rd ed.). Oxford University Press.
(51) Pollitt, C., Thiel, S. & Homburg, V. (2007). New Public Management in Europe. Management Online Review, 1-7.
(52) Sánchez, A. (2015). Public Procurement and the EU Competition Rules. (2nd ed.) Hart Publishing.
(53) Schapper, P., Veiga Malta, J. & Gilbert, D. (2006). An Analytical Framework for the Management and Reform of Public Procurement. Journal of Public Procurement, 6(1), 1-26.
(54) Secchi, L. (2009). Modelos organizacionais e reformas da administração pública. Revista de Administração Pública, 43(2), 347-369.
(55) Snider, K. & Rendon, R. (2008). Public procurement policy: Implications for theory and practice. Journal of Public Procurement, 8(3), 310-333.
(56) Snider, K. & Rendon, R. (2012). Public Procurement: Public Administration and Public Service Perspectives. Journal of Public Affairs Education, 18(2), 327-348.
(57) Tátrai, T. (2015). Debate: Cutback Management Through Public Procurement. Public Money & Management, 35(4), 255-256.
(58) Thai, K. V. (2001). Public Procurement Re-Examined. Journal of Public Procurement, 1, 9-50.
(59) Thai, K. V. (2009). International Public Procurement: Concepts and Practices. In International Handbook of Public Procurement (pp. 1-22). Taylor & Francis Group.
(60) Thai, K. V. (2017). Global Public Procurement Theories and Practices: An Introduction. In Global Public Procurement Theories and Practices (pp. 1-14). Springer International Publishing.
(61) Thai, K.V., Araujo, A., Carter, R. Y., Callender, G. & Telgen, J. (2005). Challenges in Public Procurement:an international perspective (1st ed.). PrAcademics Press.
(62) The European Parliament and the Council of the European Union. (2014). Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on Public Procurement and Repealing Directive 2004/18/EC. Official Journal of the European Union, (L 94), 65-242.
(63) The Government Shared Services Entity (eSPap). (2022). Public Procurement Shared Services. Retrieved April 10, 2022.
www.espap.gov.pt/en/spcp/Pages/spcp.aspx#maintab3
(64) Thi, T., Essig, M. & Amann, M. (2012, August 17-19). Public Procurement Benchmarking: Conceptual Model and First Empirical Findings [Paper presentation]. Fifth International Public Procurement Conference, Seattle, USA.
http://www.ippa.org/images/PROCEEDINGS/IPPC5/Part5/PAPER5-7.pdf
(65) Trepte, P. (2005). Transparency and accountability as tools for promoting integrity and preventing corruption in procurement: Possibilities and limitations. OECD Papers, 5(10), 1-37.
(66) Vaidya, K., Sajeev, A. & Callender, G. (2006). Critical factors that influence e-procurement implementation success in the public sector. Journal of Public Procurement, 6(1-3), 70-99.
(67) Walker, H. & Brammer, S. (2009). Sustainable procurement in the United Kingdom public sector. Supply Chain Management: An International Journal, 14(2), 128-137.
(68) World Bank Group. (2016). Benchmarking Public Procurement 2017. Assessing public procurement regulatory systems in 180 economies. The World Bank.
Downloads
Published
2023-01-01
How to Cite
da Silva Romão, A. L., & Ventura Ferreira, A. P. (2023). The portuguese public procurement system as a strategic instrument of public management. Tendencias, 24(1), 198–225. https://doi.org/10.22267/rtend.222302.221
Issue
Section
Research Articles
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Those authors who have publications with this journal, accept the following terms:
This journal is licensed under a Creative Commons Reconocimiento-NoComercial 4.0 Internacional License. The articles can be copied, distributed, adapted and communicated publicly, as long as the credits of the work are recognized and the respective source is quoted. This work can not be used for commercial purposes.
To increase their visibility, documents are sent to databases and indexing systems.
The content of the items is the responsibility of each author, and does not compromise in any way, journal or institution.
