← Back to leaderboard

deepseek-r1:14b

10 GB measured VRAM · 8-12GB tier
Tier 1 — floor 5/5
seed 1 (passed): all cases passed
992 tokens · 19.544s
extracted code
def is_palindrome(s):
    cleaned = ''.join([c.lower() for c in s if c.isalnum()])
    return cleaned == cleaned[::-1]
Tier 2 — edge-case handling 5/5
seed 1 (passed): all cases passed
4096 tokens · 50.812s
extracted code
def get_second_largest(numbers):
    if len(numbers) < 2:
        return None
    unique_numbers = list(set(numbers))
    sorted_unique = sorted(unique_numbers, reverse=True)
    if len(sorted_unique) < 2:
        return None
    else:
        return sorted_unique[1]
Tier 3 — multi-class interface 5/5
seed 1 (passed): all cases passed
2198 tokens · 26.999s
extracted code
class Book:

    def __init__(self, title, author, isbn, available=True):
        self.title = title
        self.author = author
        self.isbn = isbn
        self.available = available

    def __str__(self):
        return f'Title: {self.title}, Author: {self.author}, ISBN: {self.isbn}, Available: {self.available}'

class Library:

    def __init__(self):
        self.books = []

    def add_book(self, book):
        """Add a new book to the library."""
        self.books.append(book)

    def checkout_book(self, isbn):
        """Check out a book by ISBN. Returns True if successful, False otherwise."""
        for book in self.books:
            if book.isbn == isbn:
                if book.available:
                    book.available = False
                    return True
                else:
                    return False
        return False

    def return_book(self, isbn):
        """Return a checked-out book by ISBN. Returns True if successful, False otherwise."""
        for book in self.books:
            if book.isbn == isbn:
                if not book.available:
                    book.available = True
                    return True
                else:
  
... [truncated, 310 more chars]
Tier 4 — bug-fix 5/5
seed 1 (passed): all cases passed
2720 tokens · 33.871s
extracted code
def apply_discount(price, discount_percent):
    """
    Applies a percentage discount to a price.
    discount_percent is 0-100 (e.g., 20 means 20% off).
    Returns the discounted price, rounded to 2 decimal places.
    Raises ValueError if discount_percent is negative or greater than 100.
    Raises ValueError if price is negative.
    """
    if not 0 <= discount_percent <= 100:
        raise ValueError('Discount percent must be between 0 and 100')
    if price < 0:
        raise ValueError('Price cannot be negative')
    discount = price * discount_percent / 100
    new_price = price - discount
    return round(new_price, 2)
Tier 5 — schema cross-reference 0/5
seed 1 (FAILED): top_customers not defined
4096 tokens · 50.938s
Tier 6 — strict JSON format 5/5
seed 1 (passed): strict JSON, all values correct
512 tokens · 6.354s
raw response
{"low_stock_items": ["Widget", "Gizmo"], "total_value": 289.85, "most_expensive_item": "Gizmo"}