National Taipei University of Technology Uses AI-Stack for Fair GPU Resource Allocation, Encouraging Professors to Conduct Research and Foster Innovation

In the current landscape, where industries are pushing for digital transformation, there’s an increasing demand for AI talent, especially those who can apply AI technology across various sectors. To meet this need, universities with a mandate to train skilled professionals are establishing related departments and programs and promoting AI innovation research to help students build promising careers after graduation.

The National Taipei University of Technology (Taipei Tech), known for emphasizing practical applications in teaching, has been eager to strengthen its AI teaching and research capabilities. In line with the Ministry of Education’s program to optimize hands-on environments at technical colleges, Taipei Tech established the “Northern Taiwan Technical College AI School,” supporting students from over ten partner schools in developing core skills for the future AI industry. Consequently, the university must prepare substantial AI computing resources to meet both internal and external demands.

Yung-Chung Wang, Director of the Computer and Network Center at Taipei Tech, explained that after extensive planning, the center implemented a two-phase project. First, with the help of Tatung System Technologies (TST) and Nvidia, they introduced five NVIDIA DGX-1 GPU servers, each equipped with Tesla V100 graphics cards. The next crucial step was to select a high-quality collaborative AI computing resource management platform; without effective real-time and fair allocation, the entire project would fail. Following a rigorous proof-of-concept (POC) process, Taipei Tech chose the AI-Stack platform provided by INFINITX Software in May-June 2020.

Customization Needs and Strict Evaluation of Vendor Capabilities

The Computer and Network Center has long aimed to centralize information systems for enhanced cybersecurity. However, given the independent nature of academic units, this goal had yet to be realized. The AI computing platform presented a breakthrough opportunity. Director Wang explained that the platform provided critical incentives for faculty, including initial free access, a seamless experience similar to an in-house setup, relief from server maintenance burdens, and a quieter operating environment that would not disrupt learning. The center also assigned a three-tiered user role system for the platform: “primary manager” (Computer and Network Center), “project manager” (faculty), and “user” (students). Faculty members have resource management privileges, allowing the center to handle infrastructure like servers in a professional data center environment, reducing maintenance workload for departments and encouraging faculty to focus their research and project efforts on this platform to secure priority access to these resources.

This concept required a robust management platform to make it feasible. When selecting the management software, the center prioritized platforms based on the Kubernetes (K8s) architecture, as launching virtual machines consumes substantial resources, whereas containers avoid this issue and maintain computational independence. K8s, known as the best container orchestration platform, thus became a primary criterion.

After Taipei Tech released its requirements, several software providers vied for the project, each showcasing impressive management features. Director Wang noted that collaborative AI computing resource management is a novel concept, and with varying scenarios across universities, a one-size-fits-all solution is impractical. Therefore, they deemed the “technical capability” of vendors a key factor, putting POC results to the test. Taipei Tech’s POC setup was unique, with the center specifying foundational management needs such as account and password integration (single sign-on), custom image management, user-friendly environments for those unfamiliar with K8s or Linux, and secure access for external AI School users. Faculty and students were also invited to test the system, providing feedback from a user’s perspective.

Faced with challenging requirements, technical staff at Taipei Tech’s Computer and Network Center, led by Xin Zhou, meticulously compiled the center’s questions and faculty feedback, working closely with INFINITX Software to address and adjust strategies. Given INFINITX’s strong technical foundation, they responded earnestly to every question and viewed this project as a strategic partnership, appreciating Taipei Tech for providing the setting and know-how that helped enhance AI-Stack. Their commitment earned the center’s trust, leading to a swift decision to collaborate with the AI-Stack team at INFINITX Software after the POC.

Creating a User-Friendly Environment to Facilitate AI Program Development for Faculty and Students

INFINITIX’s thorough responses met Taipei Tech’s rigorous requirements with well-rounded solutions. For instance, they designed a “custom image management” feature that allows users to save their environment state, including any special packages or libraries, for easy restoration without time-consuming reconfiguration. Complex Linux commands were also simplified into one or two web-based buttons, enabling users to quickly enter the familiar Jupyter Notebook environment, bypassing platform integration challenges to focus on AI program development.

Tatung System Technologies, the project partner, demonstrated strong project management and communication skills throughout the end-to-end process, from initial requirement interviews to planning, setup, and maintenance, enabling AI-Stack to launch on time and meet quality standards by mid-August 2020. Director Wang revealed that to ensure the platform’s longevity, Taipei Tech’s president closely monitored the project’s progress. Learning that resources could be allocated fairly and that every AI teaching or research user would have sufficient resources eased his concerns.

Since its launch, AI-Stack has delivered two significant benefits for Taipei Tech. In terms of teaching support, it has facilitated AI course offerings across all six colleges, and it also meets resource demands for external AI School users. External users accessing Taipei Tech’s internal network can follow preset paths to designated servers, specific GPU cards, and dedicated storage, launch specific software, and do so without compromising campus security or privacy—a pioneering achievement in Taiwan.

On the research side, the platform’s completion has prompted the university to encourage faculty to actively engage in research projects, using Taipei Tech’s resources to develop new applications. As a result, Taipei Tech has successfully promoted AI applications in healthcare and smart manufacturing, with remarkable outcomes.