DELPHI METHOD APPLICATION FOR EVALUATION OF INNOVATIVE PRODUCT DEVELOPMENT SCENARIO

. The authors had the idea to apply the Delphi method to evaluate innovative product development scenarios. For this purpose, questionnaires were prepared, and two rounds of the investigation were carried out, in which 30 experts participated as the respondents to evaluate factors related to innovative product development scenarios. A set of the factors and sub-factors to be assessed was selected for the research. This set was formed as the basis from the paper's authors' collected and summarised data. After analysis of obtained research results, it was found that the successful introduction of an innovative product to the market, strategic agility


Introduction
For innovative product development in a rapidly changing environment, it is required to develop methods to generate possible scenarios and select the most appropriate set for the consumer and producer.The Delphi method can be used to assess a number of views of experts from various fields.The authors had the idea to use the Delphi method to create the background to develop and evaluate innovative product development scenarios.
Innovative product development must catch up on more predictable scenarios that enable increased go-tomarket success, greater strategic agility, and reduced risk.Creating scenarios in companies is limited because this activity requires considerable resources and relevant personnel competencies.The great efforts of the companies pay back the investment, as it can prevent risks that can cause more significant losses through detailed analysis of trends.Creating scenarios allows you to determine the most likely development of the business environment and to prepare for these challenges in advance by predicting the necessary business changes and resources to implement the changes.However, when quality scenarios are ready, the situation becomes much more manageable thanks to informed decisions.
The work aims to provide research on the Delphi method application possibilities in the field of innovative product development scenario creation and to establish a set of factors for making decisions for the best scenario selection.
The subject of research is the Delphi method application for evaluating innovative product development scenarios.
The main application area for the research results is engineering and marketing activity for innovative product development.The research involved Lithuanian experts of various fields to participate in the Delphi method for different scenarios evaluation.The results of the investigation can help create innovative product development scenarios.

Theoretical background
According to O' Brien and Meadows (2013), scenario planning is one of the tools consistently reported as being used by executives to support their business development.
The creation of innovative product development scenarios becomes important due to the identification and overcoming of existing barriers related to innovation development.Based on Castaneda et al. (2023), one of the most critical challenges organisations face to innovate is dealing with different types of barriers, such as demand uncertainty, product imitation, lack of employees, scarcity of government funding, and absence of internal and external financing.Anticipating future customer needs under uncertainty to elicit robust top-level design requirements is the motivation for using future scenarios (Randt, 2015).
According to Avagyan et al. (2022), firms can practically take specific actions to encourage small-range scenario presentation, often by setting expectations and/or providing resources and training.
The creation of innovative product development scenarios can be positively influenced by the application of artificial intelligence in organisational settings to enable the safety of workers as well as the ability to speed up the rate of complex global problem solving (Farrow, 2022) and the use of the digital twin method for combining systems and systems-of-systems simulations to run trade-off analysis based on different product-service systems configurations Bertonia et al. (2022).
In our study, the Delphi method is used to identify factors important for scenario evaluation.The development of future scenarios is often combined with different participatory approaches; one among many is the Delphi method, widely adopted for its systematic and interactive nature, according to Calleo and Pilla (2023).
Based on Marchais-Roubelat and Roubelat (2011), the Delphi method is important to give access to specific forms of knowledge, and this knowledge may be characterised according to the type of knowledge sought after, its status, its temporality, its field of use and the risk of bias that may affect it.According to Konu's (2015) study, customer ideas and opinions were used in new product and service development even though it has sometimes been found challenging, e.g., by criticising that customers do not necessarily know what they want.Two Delphi rounds were used in the Konu (2015) study.The first round was used to collect new ideas for different purposes in new service development.These ideas were then analysed, and thematic products/product themes were formed by using narrative analysis.During the second round, alternative forms or products were suggested.
According to Ribeiro and Quintanilla (2015), participants of the Delphi method found it challenging to assess variables and support their opinion in the absence of evidence and make judgements under briefly described scenarios; the questionnaire was considered to be long and included complex questions, making participation in the survey rather time-demanding; the design of the survey did not allow space for a debate on the positive aspects of the subject.
The Delphi method is also important to harmonise different opinions.The Delhi study of Giménez-Medina et al. (2023) reached a consensus on requirements for an Agile Innovation Funding Framework oriented toward obtaining an improved competitive advantage for information and communication technology products or services based on trust, transparency, inspection, and adaptation principles.

Research course and methodology
As the first step, the aim of the research was formulated.In the second step, the Delphi method application case research took place.The third step involved the analysis and evaluation of the obtained data by using an average value method, Kendall's coefficient of concordance and after the analysis, adequate conclusions were stated.The authors had the idea to use the Delphi method to establish the factors for innovative product development scenarios.For this purpose, questionnaires were prepared, and two rounds of investigation were carried out, in which 30 experts were the respondents: 1.During the first round of investigation, the respondents were asked about the set of factors and sub-factors for evaluation and its evaluation expediency.For this purpose, professionals from industry and science institutions were interviewed.
2. During the second round of investigation the respondents were asked to determine the weights of the factors for different categories of the scenarios.Scenarios were divided into the successful introduction of an innovative product to the market, strategic agility, and reduction of uncertainty scenarios, which are favourable.Also, the respondents were interviewed, and the priority positions of the parameters were determined, which was done according to the importance of the successful implementation of innovative product development scenarios (respondents were asked to identify an order sequence number from 1 to 12).The questionnaire was developed in order to harmonise the opinions of the experts.The results were processed using Kendall concordance coefficient.
Kendall's coefficient of concordance KW was calculated as: where ri,k is given the rank for the object i by the judge number k, nthe total number of the objects, and mthe total number of judges.If the KW is 1, all the survey respondents have assigned the same rank sequence to the list of concerns.If KW is 0, then there is no overall trend of agreement among the respondents.

Preparation for the research and results of the first Delphi round
For the first stage of Delphi, a list of factors that we selected from literature sources was offered.The results of this analysis are presented in Tables 1, 2, 3.These factors became the basis for forming the questionnaire of the first Delphi round.During the first round of Delphi, 30 respondents were interviewed.These respondents were mainly 20-29 years old (67 per cent) and 30-29 years old (14 per cent), working as engineers (37 per cent) and heads of departments or managers (23 per cent), representing industrial companies (80 per cent of respondents).Most of the respondents (80 per cent) had higher education.They were offered a list of factors that we selected from literature sources.The results of this analysis are presented in Tables 4, 5, 6.
Respondents had to assess the need for existing factors to prepare and evaluate innovative product development scenarios.Also, the respondents had to propose their factors, which are also appropriate to consider.Evaluation of the factors by points, according to the Likert scale, is presented in Tables 4, 5, 6.Large number of international markets for product distribution 3.93 3.2.A large number of international agreements for the supply of raw materials 3.5 3.3.Global availability of product information 4.47 3.4.High number of international patents used in the product 3.4 3.5.Large number of foreign countries where production is planned 3.2 3.6.Use of international labour 3 3.7.Use of international capital 3.23 3.8.Use of international high technologies 4.07 3.9.Showing the product at exhibitions abroad * 4 3.10.Transport options * 5 3.11.Advertising abroad * 4 3.12.Risk created by the spread of infectious diseases * 5 * Additional factors proposed to be included by survey respondents  6. Factors (the sustainability of the product (6th group of factors), the determination of the preliminary price (7th group of factors), the compliance with norms and standards (8th group of factors) and four additional groups of factors (9th-12th)) obtained during the first round of Delphi and their importance Factors Average points 6.1.Recyclability of product elements 3.97 6.2.Absence of adverse effects on humans 4.7 6.3.Absence of negative impact on nature 4.7 6.4.Production from renewable sources 3.7 6.5.Environmental friendliness of operation 4.07 6.6.Environmental friendliness of production 4.07 6.7.Environmental friendliness of transportation 3.9 6.8.Necessity of the product for the consumers * 5 6.9.Warranties for the product, service * 4.5 7.1.For the successful implementation of an innovative product, it is important to determine its preliminary price 4.43 8.1.For the successful implementation of an innovative product, it is important whether the product complies with norms and standards 4.37 After the first round of the survey, the list of analysed factors was supplemented with 32 factors proposed by the respondents of the surveyed companies.Also, the following groups of factors (9th-12th) have been added to the list: the management's policy towards the innovative product, brand distribution, the situation of preparation and implementation of the innovative product project, and the possibilities to change personnel.

Results of the second Delphi round and their application for scenario development
12 respondents participated in the second round.These respondents mainly were 20-29 years old (42 per cent) and 30-29 years old (33 per cent), working as department heads and managers (33 per cent), engineers (17 per cent), directors (17 per cent), and representing industrial companies.(50 percent of respondents).Most of the respondents (92 per cent) had higher education.
Using the basis of the finer factors, the coarser factors, which were weighted in the second stage of the Delphi method, were constructed for the three main scenarios: 1.How important are each of the following factors for the successful introduction of an innovative product to the market?2. How much attention and resources should be devoted to each factor below to ensure strategic agility during an innovative product launch?3. How much attention and resources should be devoted to each factor below to reduce uncertainty (risk) for introducing an innovative product?
These three scenarios were selected based on the literature that emphasised the importance of the relevant factors for product innovation.Ding and Ding (2022) stated that technological innovativeness and market innovativeness are both associated with perceived new product performance, and market innovativeness is more pertinent to new product performance than technological innovativeness for new ventures.Nathan and Rosso's (2022) paper sheds new light on the links between firm-level innovation and growth.Walheiser et al. (2021) suggest that the translation of firm-level product innovativeness into successful commercialisation of new products is facilitated when firms' organisational structures are designed to unleash their abilities to overcome internal resistances in the innovating organisation and external resistances in the marketplace.
The results of Mata et al. (2023) study indicate that companies are more willing to acquire knowledge from external environments (customers, competitors, markets, etc.); transforming this information efficiently leads to new services and products, improving innovation and boosting the success rate of projects in return, and strategic agility can provide further information to help organisations reform and renew strategically.According to Tarba et al. (2023), strategic agility is a vital asset enabling firms to cope with an uncertain and changing world.
According to Ivory and Brooks (2018), the application of strategic agility to managing corporate sustainability with a paradoxical lens can comprise three organisational meta-capabilities: strategic sensitivity, collective commitment, and resource fluidity.
Based on Helm and Gritsch (2014), international entrepreneurship has a more significant impact on uncertainty reduction than the use of networks, and after having reduced uncertainty, a firm tends to adapt its communication and pricing strategy, whereas the adaptation of the product and distribution strategy, in general, is not significant.
After analysing the received data and applying the Kendall concordance coefficient calculation methodology, it was found that there is a significant difference in the experts' opinions on what parameters should be a priority.The concordance rate of 0.22 was obtained when the maximum value is 1.
The obtained results are presented in Figure 1.As shown in Figure 1, for all three scenarios, the factors are evaluated quite similarly; the factors related to product innovativeness and the factors related to management policy stand out a little.There is a significant difference of several points between some factors, such as the possibility of changing employees and the innovativeness of the product.The obtained results can be used to develop and evaluate innovative product development scenarios.This process is shown in Figure 2. Based on the literature analysis (Hassan et al., 2012;Mazzei et al., 2016;Hallstedt, 2017;Sivam et al., 2019;Rezk et al., 2019;Wang et al., 2021;Ma et al., 2021;Walheiser et al. 2021;Avagyan et al., 2022;Zamborský et al., 2023;Giménez-Medina et al., 2023;Castaneda et al., 2023), it was established that many authors agree that the development of an innovative product begins with the determination of market demand and its trends, or with the product development process initiated within the company itself.Such analysis should show which products are competitive in the market and which are not.Products that are uncompetitive in the market must be additionally evaluated to determine whether they can still be updated with the help of innovation or not.Only for products that still have a good development future an innovation process is initiated, which can be entirely dictated and implemented by the company developing innovations (Push strategy) or focusing on the dynamic environment of market demand; the innovation process can be organised and implemented taking into account specific individualised market needs (Pull strategy).For both strategies, it is sometimes possible to develop sufficiently defined scenarios for their development.If this is possible, then industrial companies can and should follow this path to detail the market development alternatives, link them with industrial product innovation alternatives, and more accurately assess the risks and their management decisions.The results of the empirical study showed that the most relevant are the scenarios of introduction to the market, Strategic agility during the introduction of the product to the market, and Uncertainty risk reduction, which best meet the expectations of industrial companies and the market situation for creation the innovative product and development of it.Considering the selected and approved scenarios, industrial companies implement them by purposefully creating and developing innovative products.This process must be implemented with a continuous assessment of the market situation and the innovative process itself so that, if necessary, the relevant adjustment decisions can be made.As a result, the created product, as an innovative product, is launched to the market with the most evaluated scenario of its development, maximising success and minimising risks.

Discussion
Creating innovative product development scenarios can be helpful for companies that want sustainable longterm development in the presence of considerable uncertainty and applying the principles of strategic agility.That the creation and implementation of innovative product development scenarios is taking place is evident from the results of our research.As shown by the survey carried out in the first round of the Delphi method, 83 per cent of the respondents of the surveyed companies indicated that their companies produced innovative products.In comparison, 17 per cent of the respondents indicated that they did not.Also, 43 per cent of the respondents stated that their company creates scenarios for the development of innovative products, 33 per cent that it does not, and 24 per cent of the respondents did not know about it.When asked who usually develops innovative products, 60 per cent of respondents indicated that their company with customers, 13 per cent that only their company, 10 per cent with suppliers, 14 per cent together with business partners or that scenarios are developed only by business partners, 3 per cent of respondents knew nothing about it.When asked who usually implements innovative product development scenarios, 20 per cent of respondents indicated that their company with customers, 27 per cent that only their company, 20 per cent with suppliers, 13 per cent together with business partners or that scenarios are implemented only by business partners, 20 per cent of respondents did not know anything about it.As can be seen from the survey, when creating innovative products, companies cooperate strongly with consumers and other business partners.In this area, greater cooperation is possible in developing innovative products and their scenarios.
By examining the three possible priorities of the parameters of the innovative product development scenarios, it is possible to narrow down the various scenarios and select the ones with the highest scores.Differences of opinion and disagreements can be explained by the fact that the business environment is challenging to predict, and there is a lack of competencies in predicting business processes.Critical factors (driving forces) that were determined and evaluated for the creation and choosing of possible scenarios -product introduction to the market, strategic agility, or uncertainty (risk) reduction, may have a different influence on the success of innovative product development.When writing scenarios, one has to anticipate the effects of these factors and their possible reactions to them to achieve the intended results.It is also worth considering the mutual interaction of factors and their change trends during the period for which the scenario is being developed.
It is worth considering the concept of green innovation because, as the research of Banelienė and Strazdas (2023) shows, green innovations have a positive impact on economic growth in the European Union.Thao and Xie (2023) also suggest promoting the efficiency of green innovation through an open innovation strategy.Farooq et al. (2024) also state that with rising pollution emissions, it is vital to devise regulatory policies that ensure sustainable development and green innovation offers an alternative strategy, fostering economic progress and environmental sustainability.In our scenarios, green innovation can partially cover several groups of factors, such as groups of product sustainability and compliance with norms and standards.
It should also be noted that scripting is labour-intensive, and our proposed method can help reduce time costs.
In addition, finding a consensus in evaluating various factors is essential because the study showed a considerable difference in the respondents' opinions.
Labour intensity can be reduced with the help of artificial intelligence (AI).Creating a scenario using AI requires formulating prompts that specify the objective, the factors, the query about their impact, and the planned response to that impact.After reviewing many factors, it is possible to summarise everything and combine the text into a coherent operational scenario, avoiding conflicting, irrelevant or illogical decisions.

Conclusions
After application of the Delphi method to evaluate the innovative product development scenario, the following conclusions were stated: 1.After reviewing the sources of scientific literature, it was found that creating scenarios can provide advantages in reducing business uncertainty and facilitating the innovation development process.The application of the Delphi method enables gathering information about the factors important for developing innovative product scenarios and evaluating them to select the most suitable scenarios.Based on literature sources, the factors that formed the basis for the Delphi method surveys were selected.
2. The first Delphi round helped to supplement the list of factors and their groups.Respondents indicated the importance of factors using a Likert scale (from 1 to 5 points).Most factors are deemed necessary, but only a few are rated moderately important.It was also seen that applying the Delphi method and creating scenarios is a labour-intensive process that requires sound knowledge and skills of the organisers in the field under study.
3. In the second round of Delphi, weights were determined for various factors, comparing 3 possible scenarios innovative product introduction to the market, strategic agility, and uncertainty reduction.The conducted studies showed that when evaluating the weights of groups of factors in relation to all three possible scenarios, very similar results were obtained, which makes it possible to move from one scenario to another more easily during their implementation.It was also found that there is a considerable difference in the opinions of the respondents who participated in the Delphi survey.After evaluating the difference of opinion, it was determined that Kendall's concordance coefficient would be 0.22, with a maximum value of 1 and a minimum value of 0. Therefore, when evaluating prepared scenarios, it is valuable to rely on the opinions of several experts by taking the averages of their evaluations.

9. 1 .
The management policy towards the innovative product (motivation and support) ** 5 10.1.Brand distribution ** 5 11.1.Compliance with the situation of preparation and implementation of the innovative product project ** 4.33 12.1.Possibilities to change personnel ** 5 * Additional factors proposed to be included by survey respondents ** Additional groups of factors proposed to be included by survey respondents

Figure 1 .
Figure 1.The results obtained during the second round of Delphi

Figure 2 .
Figure 2. A model of innovative product development and implementation using scenarios of the current situation with the company's product portfolio No Yes Can the innovation process improve the situation with products?Eliminate the product Highlight which products do not meet the requirements Does the current situation meet the company's requirements for development and competitiveness?Yes No Assessment of which scenario (Introduction to the market; Strategic agility during the introduction of the product to the market; Reduction of the risk of uncertainty) best meets the company's expectations and the market situation for the creation and development of an innovative product Initiating the innovation process for product development and using Push or Pull strategies Can we use a scenario model to create and develop an innovation?
4. Based on the research results, an innovative product development and implementation model was created using scenarios.This model describes a possible scenario development path choice, scenario development, evaluation, selection, adaptation and implementation.5.The development and implementation of scenarios open up new possibilities for increasing the sustainability of industrial enterprises in innovative product development and introduction to the market.Having explicit scenarios allows for a more correct and effective reaction to environmental changes and acquiring a strategic advantage.

Table 1 .
Relationship between factors (the innovativeness of company employees (1st group of factors), product (2nd group of factors), and the globality of the product (3rd group of factors)) studied in this article and factors studied by various authors

Table 2 .
Relationship between factors (the product improvement process (4th group of factors) and the adequacy of the product to the environment (5th group of factors)) studied in this article and parameters studied by various authors The factors investigated in the current articleThe authors in whose articles similar factors were investigated 4.1.Improvement that is based on creativity Tennyson & Breuer (2002) 4.2.Improvement is based on problem-solving Tennyson, & Breuer (2002) 4.3.Improvement is based on replication of successful cases Tennyson, & Breuer (2002) 5.1.Compliance of the product with the overall strategy of the company Hallstedt (2017) 5.2.Compliance of the product with the "push" strategy Urbaniec & Żur (2021).5.3.Compliance of the product with the "pull" strategy Urbaniec & Żur (2021).5.4.Compliance with the needs of the company's interest groups Wang et al. (

Table 3 .
The relationship between factors (the sustainability of the product (6th group of factors), the determination of the preliminary price (7th group of factors), and the compliance with norms and standards (8th group of factors)) studied in this article and parameters studied by various authors Wang et al. (2021)Filho (2022)ntation of an innovative product, it is important whether the product complies with norms and standardsHallstedt (2017),Brandao & Godinho-Filho (2022),Wang et al. (2021)

Table 4 .
Factors (the innovativeness of company employees (1st group of factors), product (2nd group of factors), and the globality of the product (3rd group of factors)) obtained during the first round of Delphi and their importance

Table 5 .
Factors (the product improvement process (4th group of factors) and the adequacy of the product to the environment (5th group of factors)) obtained during the first round of Delphi and their importance